{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Code for Feedforward Quantum Neural Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 Package-imports, universal definitions and remarks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the relevant packages:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#math related packages\n",
    "import scipy as sc\n",
    "import qutip as qt\n",
    "#further packages\n",
    "from time import time\n",
    "from random import sample\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Declaration of qubit kets and density matrices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#ket states\n",
    "qubit0 = qt.basis(2,0)\n",
    "qubit1 = qt.basis(2,1)\n",
    "#density matrices\n",
    "qubit0mat = qubit0 * qubit0.dag()\n",
    "qubit1mat = qubit1 * qubit1.dag()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Declaring functions for Dirac notation:\n",
    "\n",
    "$$\\newcommand{\\ket}[1]{\\left|{#1}\\right\\rangle}$$\n",
    "$$\\newcommand{\\bra}[1]{\\left\\langle{#1}\\right|}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data conventions\n",
    "\n",
    "The parameter QNN-Architecture $\\text{qnnArch}$ describes the structure of a quantum neural network. It is expected to be a 1-dimensional list of natural numbers that which refer to the number of perceptrons in the corresponding layer. E.g. a 2-3-2 network would be given by $\\text{qnnArch = [2, 3, 2]}$\n",
    "\n",
    "\n",
    "Any parameter $\\text{unitaries}$ is expected to be a 2-dimensional list of the networks perceptron unitaries (qutip.Qobj) given in a tensored state like\n",
    "\n",
    "$$\\text{unitaries}  = \\big[ [\\, ],[U_1^1, \\dots, U_{m_1}^1], \\dots, [U_1^l, \\dots, U_{m_l}^l], \\dots, [U_1^{out}, \\dots, U_{m_{out}}^{out}]\\big]$$\n",
    "\n",
    "$$\\text{unitaries[l][j]} = U_{j+1}^l$$\n",
    "\n",
    "(The latter formula assumes that zero-indexing is not used for the perceptron unitaries, as is indicated in the former formula)\n",
    "\n",
    "\n",
    "Any parameter $\\text{trainingData}$ is expected to be a 2-dimensional list, where every inner list has exactly two elements like\n",
    "\n",
    "$$trainingData = \\big[ [\\text{state}_1,\\, \\text{unitary} \\cdot \\text{state}_1], \\dots , [\\text{state}_N,\\, \\text{unitary} \\cdot \\text{state}_N] \\big]$$\n",
    "\n",
    "where $\\text{unitary}$ is the targeted network unitary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 Helper functions for the QNN-Code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Partial Trace\n",
    "\n",
    "The $\\text{partialTrace}$ functions assume to be given a quantum object (qutip.Qobj) $\\text{obj}$ that is the tensor product of m factors $\\{0, 1, \\dots, m-1\\}$. The functions then calculate and return the partial trace as follows:\n",
    "\n",
    "$\\text{partialTraceRem}$ traces out the factors $$\\text{rem} \\subsetneq \\{0, \\dots, m-1\\}$$\n",
    "\n",
    "$\\text{partialTraceKeep}$ traces out the factors $$\\big( \\{0, \\dots, m-1\\} \\setminus \\text{keep}\\big) \\subsetneq \\{0, \\dots, m-1\\}$$\n",
    "\n",
    "No duplicates in $\\text{rem}$ allowed. No values $\\geq m$ in $\\text{rem}$ or $\\text{keep}$ allowed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def partialTraceRem(obj, rem):\n",
    "    #prepare keep list\n",
    "    rem.sort(reverse=True)\n",
    "    keep = list(range(len(obj.dims[0])))\n",
    "    for x in rem:\n",
    "        keep.pop(x)\n",
    "    res = obj;\n",
    "    #return partial trace:\n",
    "    if len(keep) != len(obj.dims[0]):\n",
    "        res = obj.ptrace(keep);\n",
    "    return res;\n",
    "\n",
    "def partialTraceKeep(obj, keep):\n",
    "    #return partial trace:\n",
    "    res = obj;\n",
    "    if len(keep) != len(obj.dims[0]):\n",
    "        res = obj.ptrace(keep);\n",
    "    return res;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basis swap\n",
    "\n",
    "$\\text{swappedOp}$ assumes to be given a quantum object (qutip.Qobj) $\\text{obj}$ that is the tensor product of $m$ factors. For\n",
    "\n",
    "$$\\text{obj} = A_0 \\otimes \\dots \\otimes A_i \\otimes \\dots \\otimes A_j \\otimes \\dots A_{m-1}$$\n",
    "\n",
    "the function returns\n",
    "\n",
    "$$A_0 \\otimes \\dots \\otimes A_j \\otimes \\dots \\otimes A_i \\otimes \\dots A_{m-1}$$\n",
    "\n",
    "where $i, j \\in \\{1, \\dots, m \\}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def swappedOp(obj, i, j):\n",
    "    if i==j: return obj\n",
    "    numberOfQubits = len(obj.dims[0])\n",
    "    permute = list(range(numberOfQubits))\n",
    "    permute[i], permute[j] = permute[j], permute[i]\n",
    "    return obj.permute(permute)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Further functions\n",
    "\n",
    "$\\text{tensoredId}$ returns $\\underbrace{\\mathbb{1}_2 \\otimes \\dots \\otimes \\mathbb{1}_2}_{N \\text{-times}}$\n",
    "\n",
    "$\\text{tensoredQubit0}$ returns $\\ket{0 \\dots 0}_N \\bra{0 \\dots 0}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def tensoredId(N):\n",
    "    #Make Identity matrix\n",
    "    res = qt.qeye(2**N)\n",
    "    #Make dims list\n",
    "    dims = [2 for i in range(N)]\n",
    "    dims = [dims.copy(), dims.copy()]\n",
    "    res.dims = dims\n",
    "    #Return\n",
    "    return res\n",
    "\n",
    "def tensoredQubit0(N):\n",
    "    #Make Qubit matrix\n",
    "    res = qt.fock(2**N).proj() #For some reason ran faster than fock_dm(2**N) in tests\n",
    "    #Make dims list\n",
    "    dims = [2 for i in range(N)]\n",
    "    dims = [dims.copy(), dims.copy()]\n",
    "    res.dims = dims\n",
    "    #Return\n",
    "    return res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\text{unitariesCopy}$ assumes to be given $\\text{unitaries}$ as usual and returns a deep copy of it, meaning a new 2-dimensional list where every element is a copy of the element at the same position in the given 2d-list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def unitariesCopy(unitaries):\n",
    "    newUnitaries = []\n",
    "    for layer in unitaries:\n",
    "        newLayer = []\n",
    "        for unitary in layer:\n",
    "            newLayer.append(unitary.copy())\n",
    "        newUnitaries.append(newLayer)\n",
    "    return newUnitaries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 Random generation of unitaries, training data and networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generation of random unitaries:\n",
    "\n",
    "$\\text{randomQubitUnitary}$ return a unitary $2^{\\text{numQubits}} \\times 2^{\\text{numQubits}}$-matrix as a quantum object (qutip.Qobj) that is the tensor product of $\\text{numQubits}$ factors. Before orthogonalization, it's elements are randomly picked out of a normal distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def randomQubitUnitary(numQubits):\n",
    "    dim = 2**numQubits\n",
    "    #Make unitary matrix\n",
    "    res = sc.random.normal(size=(dim,dim)) + 1j * sc.random.normal(size=(dim,dim))\n",
    "    res = sc.linalg.orth(res)\n",
    "    res = qt.Qobj(res)\n",
    "    #Make dims list\n",
    "    dims = [2 for i in range(numQubits)]\n",
    "    dims = [dims.copy(), dims.copy()]\n",
    "    res.dims = dims\n",
    "    #Return\n",
    "    return res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generation of random training data\n",
    "\n",
    "$\\text{randomQubitState}$ return a unitary $2^{\\text{numQubits}}$-vector as a quantum object (qutip.Qobj) that is the tensor product of $\\text{numQubits}$ factors. Before normalization, it's elements are randomly picked out of a normal distribution.\n",
    "\n",
    "$\\text{randomTrainingData}$ assumes to be given a unitary quantom object (qutip.Qobj) $\\text{unitary}$ that is the tensor product of m factors and returns a list of $N$ training pairs like\n",
    "\n",
    "$$trainingData = \\big[ [\\text{state}_1,\\, \\text{unitary} \\cdot \\text{state}_1], \\dots , [\\text{state}_N,\\, \\text{unitary} \\cdot \\text{state}_N] \\big]$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def randomQubitState(numQubits):\n",
    "    dim = 2**numQubits\n",
    "    #Make normalized state\n",
    "    res = sc.random.normal(size=(dim,1)) + 1j * sc.random.normal(size=(dim,1))\n",
    "    res = (1/sc.linalg.norm(res)) * res\n",
    "    res = qt.Qobj(res)\n",
    "    #Make dims list\n",
    "    dims1 = [2 for i in range(numQubits)]\n",
    "    dims2 = [1 for i in range(numQubits)]\n",
    "    dims = [dims1, dims2]\n",
    "    res.dims = dims\n",
    "    #Return\n",
    "    return res\n",
    "\n",
    "def randomTrainingData(unitary, N):\n",
    "    numQubits = len(unitary.dims[0])\n",
    "    trainingData=[]\n",
    "    #Create training data pairs\n",
    "    for i in range(N):\n",
    "        t = randomQubitState(numQubits)\n",
    "        ut = unitary*t\n",
    "        trainingData.append([t,ut])\n",
    "    #Return\n",
    "    return trainingData"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Note regarding Qobj.dims"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Quantum objects $\\text{Qobj}$ in the $\\text{qutip}$ package have two properties regarding their sizes: $\\text{shape}$ and $\\text{dims}$. While $\\text{shape}$ denotes the \"dimensions of the underlying data matrix\", $\\text{dims}$ is a \"list keeping track of shapes for individual components of a multipartite system\" (quoted from the $\\text{qutip}$-documentation). The qutip package will only allow operations such as the multiplication of two $\\text{Qobj}$ objects $A$ and $B$, if not only their $\\text{shape}$s, but also their $\\text{dims}$ are compatible. Essentially all $\\text{Qobj}$ objects in this work are representing operators or states in an multipartite system of $n$ qubits. As we create such objects with the desired $\\text{shape}$, we have to ensure that their $\\text{dims}$ are reflective of that, which is why we manually overwrite them to meet this criteria."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generation of an initial network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\text{randomNetwork}$ will, given a valid $\\text{qnnArch}$ as usual, return a 4-tuple that completely describes a trainable quantum-neural-network like\n",
    "\n",
    "$$\\big(\\text{qnnArch}, \\ \\text{unitaries} , \\ \\text{trainingData}, \\ \\text{networkUnitary} \\big)$$\n",
    "\n",
    "With $\\text{qnnArch},\\, \\text{unitaries}$ and $\\text{trainingData}$ as usual and $\\text{networkUnitary}$ being the targeted network unitary (with which the training data was created)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def randomNetwork(qnnArch, numTrainingPairs):\n",
    "    assert qnnArch[0]==qnnArch[-1], \"Not a valid QNN-Architecture.\"\n",
    "    \n",
    "    #Create the targeted network unitary and corresponding training data\n",
    "    networkUnitary = randomQubitUnitary(qnnArch[-1])\n",
    "    networkTrainingData = randomTrainingData(networkUnitary, numTrainingPairs)\n",
    "    \n",
    "    #Create the initial random perceptron unitaries for the network\n",
    "    networkUnitaries = [[]]\n",
    "    for l in range(1, len(qnnArch)):\n",
    "        numInputQubits = qnnArch[l-1]\n",
    "        numOutputQubits = qnnArch[l]\n",
    "        \n",
    "        networkUnitaries.append([])\n",
    "        for j in range(numOutputQubits):\n",
    "            unitary = randomQubitUnitary(numInputQubits+1)\n",
    "            if numOutputQubits-1 != 0: \n",
    "                unitary = qt.tensor(randomQubitUnitary(numInputQubits+1), tensoredId(numOutputQubits-1))\n",
    "                unitary = swappedOp(unitary, numInputQubits, numInputQubits + j)\n",
    "            networkUnitaries[l].append(unitary)\n",
    "    \n",
    "    #Return\n",
    "    return (qnnArch, networkUnitaries, networkTrainingData, networkUnitary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.4 QNN-Code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cost Function\n",
    "\n",
    "$\\text{costFunction}$ assumes to be given $\\text{trainingData}$ and $\\text{outputStates}$ so that\n",
    "\n",
    "$$\\text{trainingData[x][1]} = \\ket{\\phi_x^{out}}$$\n",
    "\n",
    "$$\\text{outputStates[x]} = \\rho_x^{out}(s)$$\n",
    "\n",
    "and computes the cost function:\n",
    "\n",
    "$$C(s) = \\frac{1}{N} \\sum_{x = 1}^N \\bra{\\phi_x^{out}} \\rho_x^{out}(s) \\ket{\\phi_x^{out}}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def costFunction(trainingData, outputStates):\n",
    "    costSum = 0\n",
    "    for i in range(len(trainingData)):\n",
    "        costSum += trainingData[i][1].dag() * outputStates[i] * trainingData[i][1]\n",
    "    return costSum.tr()/len(trainingData)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Layer-Channel & Adjoint-Layer-Channel\n",
    "\n",
    "$\\text{makeLayerChannel}$ assumes to be given $\\text{qnnArch}$ and $\\text{currentUnis}$ as usual and calculates\n",
    "\n",
    "\\begin{align}\n",
    "\\\\\n",
    "\\xi_s^l(X^{l-1}) & = \\text{tr}_{l-1}\\bigg( U^l(s) \\ \\big( X^{l-1} \\otimes \\ket{0 \\dots 0}_l \\bra{0 \\dots 0} \\big) \\ U^l(s)^{\\dag} \\bigg) \\\\\n",
    "& = \\text{tr}_{l-1}\\bigg( U_{m_l}^l(s) \\dots U_1^l(s) \\ \\big( X^{l-1} \\otimes \\ket{0 \\dots 0}_l \\bra{0 \\dots 0} \\big) \\ U_1^l(s)^{\\dag} \\dots U_{m_l}^l(s)^{\\dag} \\bigg) \n",
    "\\\\\n",
    "\\end{align}\n",
    "\n",
    "for $\\text{inputState} = X^{l-1}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def makeLayerChannel(qnnArch, unitaries, l, inputState):\n",
    "    numInputQubits = qnnArch[l-1]\n",
    "    numOutputQubits = qnnArch[l]\n",
    "\n",
    "    #Tensor input state\n",
    "    state = qt.tensor(inputState, tensoredQubit0(numOutputQubits))\n",
    "\n",
    "    #Calculate layer unitary\n",
    "    layerUni = unitaries[l][0].copy()\n",
    "    for i in range(1, numOutputQubits):\n",
    "        layerUni = unitaries[l][i] * layerUni\n",
    "\n",
    "    #Multiply and tensor out input state\n",
    "    return partialTraceRem(layerUni * state * layerUni.dag(), list(range(numInputQubits)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\text{makeAdjointLayerChannel}$ assumes to be given $\\text{qnnArch}$ and $\\text{unitaries}$ as usual and calculates\n",
    "\n",
    "\\begin{align}\n",
    "\\\\\n",
    "\\mathcal{F}_s^l(X^{l}) & = \\text{tr}_{l}\\bigg( \\big( \\mathbb{1}_{l-1} \\otimes \\ket{0 \\dots 0}_l \\bra{0 \\dots 0} \\big) \\ U^{l}(s)^{\\dag} \\ \\big( \\mathbb{1}_{l-1} \\otimes X^l \\big) \\ U^l(s) \\bigg) \\\\\n",
    "& = \\text{tr}_{l}\\bigg( \\big( \\mathbb{1}_{l-1} \\otimes \\ket{0 \\dots 0}_l \\bra{0 \\dots 0} \\big) \\ U_1^l(s)^{\\dag} \\dots U_{m_l}^l(s)^{\\dag} \\ \\big( \\mathbb{1}_{l-1} \\otimes X^l \\big) \\ U_{m_l}^l(s) \\dots U_1^l(s) \\bigg)\n",
    "\\\\\n",
    "\\end{align}\n",
    "\n",
    "\n",
    "for $\\text{inputState} = X^{l}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def makeAdjointLayerChannel(qnnArch, unitaries, l, outputState):\n",
    "    numInputQubits = qnnArch[l-1]\n",
    "    numOutputQubits = qnnArch[l]\n",
    "    \n",
    "    #Prepare needed states\n",
    "    inputId = tensoredId(numInputQubits)\n",
    "    state1 = qt.tensor(inputId, tensoredQubit0(numOutputQubits))\n",
    "    state2 = qt.tensor(inputId, outputState)\n",
    "\n",
    "    #Calculate layer unitary\n",
    "    layerUni = unitaries[l][0].copy()\n",
    "    for i in range(1, numOutputQubits):\n",
    "        layerUni = unitaries[l][i] * layerUni\n",
    "    \n",
    "    #Multiply and tensor out output state\n",
    "    return partialTraceKeep(state1 * layerUni.dag() * state2 * layerUni, list(range(numInputQubits)) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Feedforward\n",
    "\n",
    "$\\text{feedforward}$ assumes to be given $\\text{qnnArch},\\, \\text{unitaries}$ and $\\text{trainingData}$ as usual, carries out the feedforward process as described in Appendix D.2 of ***Efficient Learning for Deep Quantum Neural Networks***:\n",
    "\n",
    "For each element $\\big[\\, \\ket{\\phi_x^{in}},\\, \\ket{\\phi_x^{out}} \\big]$ in $\\text{trainingData}$ do:\n",
    "\n",
    "- Calculate the network input $\\rho_x^{in} = \\ket{\\phi_x^{in}} \\bra{\\phi_x^{in}}$\n",
    "\n",
    "- For every layer $l$ in $\\text{qnnArch}$ do:\n",
    "\n",
    "    - Apply the layer channel $\\xi_s^l$ to the output of the previous layer $l-1$\n",
    "\n",
    "    - Store the result $\\rho_x^l(s)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def feedforward(qnnArch, unitaries, trainingData):\n",
    "    storedStates = []\n",
    "    for x in range(len(trainingData)):\n",
    "        currentState = trainingData[x][0] * trainingData[x][0].dag()\n",
    "        layerwiseList = [currentState]\n",
    "        for l in range(1, len(qnnArch)):\n",
    "            currentState = makeLayerChannel(qnnArch, unitaries, l, currentState)\n",
    "            layerwiseList.append(currentState)\n",
    "        storedStates.append(layerwiseList)\n",
    "    return storedStates"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Update matrix\n",
    "\n",
    "$\\text{makeUpdateMatrix}$ assumes to be given $\\text{qnnArch},\\, \\text{unitaries},\\, \\text{trainingData}$ and $\\text{storedStates}$ as usual, as well as $\\lambda,\\, \\epsilon \\in \\mathbb{R}$, and calculates\n",
    "\n",
    "$$\\text{exp}\\big( i \\, \\epsilon \\, K_j^l(s) \\big)$$\n",
    "\n",
    "with\n",
    "\n",
    "$$K_j^l(s) = \\frac{2^{n_{a_1, \\dots, \\beta}} \\, i}{2 N \\lambda} \\sum_x \\text{tr}_\\text{rest} M_j^l(s)$$\n",
    "\n",
    "with \n",
    "\n",
    "$$M_j^l(s) = \\big[A_j^l(s), B_j^l(s) \\big]$$\n",
    "\n",
    "with\n",
    "\n",
    "\\begin{align}\n",
    "A_j^l(s) & = U_j^l(s) \\dots U_1^l(s) \\, \\big( \\rho_x^{l-1}(s) \\otimes \\ket{0 \\dots 0}_l \\bra{0 \\dots 0} \\big) \\, U_1^l(s)^{\\dag} \\dots U_j^l(s)^{\\dag} \\\\\n",
    "\\\\\n",
    "B_j^l(s) & = U_{j+1}^l(s)^{\\dag} \\dots U_{m_l}^l(s)^{\\dag} \\, \\big( \\mathbb{1}_l \\otimes \\sigma_x^l(s) \\big) \\, U_{m_l}^l(s) \\dots U_{j+1}^l(s)\n",
    "\\end{align}\n",
    "\n",
    "with\n",
    "\n",
    "$$\\sigma_x^l(s) = \\mathcal{F}_s^{l+1} \\big( \\dots \\mathcal{F}_s^{out} \\big( \\ket{\\phi_x^{out}} \\bra{\\phi_x^{out}} \\big) \\dots \\big)$$\n",
    "\n",
    "$\\text{updateMatrixFirstPart}$ calculates $A_j^l(s)$, $\\text{updateMatrixSecondPart}$ calculates $B_j^l(s)$.\n",
    "\n",
    "$\\text{makeUpdateMatrixTensored}$ tensors the calculated as above update matrix in such a way, that it can be applied to the already tensored perceptron unitaries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def makeUpdateMatrix(qnnArch, unitaries, trainingData, storedStates, lda, ep, l, j):\n",
    "    numInputQubits = qnnArch[l-1]\n",
    "    \n",
    "    #Calculate the sum:\n",
    "    summ = 0\n",
    "    for x in range(len(trainingData)):\n",
    "        #Calculate the commutator\n",
    "        firstPart = updateMatrixFirstPart(qnnArch, unitaries, storedStates, l, j, x)\n",
    "        secondPart = updateMatrixSecondPart(qnnArch, unitaries, trainingData, l, j, x)\n",
    "        mat = qt.commutator(firstPart, secondPart)\n",
    "        \n",
    "        #Trace out the rest\n",
    "        keep = list(range(numInputQubits))\n",
    "        keep.append(numInputQubits + j)\n",
    "        mat = partialTraceKeep(mat, keep)\n",
    "        \n",
    "        #Add to sum\n",
    "        summ = summ + mat\n",
    "\n",
    "    #Calculate the update matrix from the sum\n",
    "    summ = (-ep * (2**numInputQubits)/(lda*len(trainingData))) * summ\n",
    "    return summ.expm()\n",
    "\n",
    "\n",
    "def updateMatrixFirstPart(qnnArch, unitaries, storedStates, l, j, x):\n",
    "    numInputQubits = qnnArch[l-1]\n",
    "    numOutputQubits = qnnArch[l]\n",
    "    \n",
    "    #Tensor input state\n",
    "    state = qt.tensor(storedStates[x][l-1], tensoredQubit0(numOutputQubits))\n",
    "    \n",
    "    #Calculate needed product unitary\n",
    "    productUni = unitaries[l][0]\n",
    "    for i in range(1, j+1):\n",
    "        productUni = unitaries[l][i] * productUni\n",
    "    \n",
    "    #Multiply\n",
    "    return productUni * state * productUni.dag()\n",
    "\n",
    "\n",
    "def updateMatrixSecondPart(qnnArch, unitaries, trainingData, l, j, x):\n",
    "    numInputQubits = qnnArch[l-1]\n",
    "    numOutputQubits = qnnArch[l]\n",
    "    \n",
    "    #Calculate sigma state\n",
    "    state = trainingData[x][1] * trainingData[x][1].dag()\n",
    "    for i in range(len(qnnArch)-1,l,-1):\n",
    "        state = makeAdjointLayerChannel(qnnArch, unitaries, i, state)\n",
    "    #Tensor sigma state\n",
    "    state = qt.tensor(tensoredId(numInputQubits), state)\n",
    "    \n",
    "    #Calculate needed product unitary\n",
    "    productUni = tensoredId(numInputQubits + numOutputQubits)\n",
    "    for i in range(j+1, numOutputQubits):\n",
    "        productUni = unitaries[l][i] * productUni\n",
    "        \n",
    "    #Multiply\n",
    "    return productUni.dag() * state * productUni\n",
    "\n",
    "\n",
    "def makeUpdateMatrixTensored(qnnArch, unitaries, lda, ep, trainingData, storedStates, l, j):\n",
    "    numInputQubits = qnnArch[l-1]\n",
    "    numOutputQubits = qnnArch[l]\n",
    "    \n",
    "    res = makeUpdateMatrix(qnnArch, unitaries, lda, ep, trainingData, storedStates, l, j)\n",
    "    if numOutputQubits-1 != 0:\n",
    "        res = qt.tensor(res, tensoredId(numOutputQubits-1))\n",
    "    return swappedOp(res, numInputQubits, numInputQubits + j)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training of the Quantum Neural Network\n",
    "\n",
    "$\\text{qnnTraining}$ assumes to be given $\\text{qnnArch},\\, \\text{initialUnitaries}$ and $\\text{trainingData}$ as usual, as well as $\\lambda,\\, \\epsilon \\in \\mathbb{R}$ and $\\text{trainingRounds} \\in \\mathbb{N}$, and trains the given quantum neural network as summarized in Figure 2 of ***Efficient Learning for Deep Quantum Neural Networks***:\n",
    "\n",
    "**Step 1: Feedforward:** For every training pair, for every layer: Apply the layer channel $\\xi^l$ and store the result\n",
    "\n",
    "**Step 2: Updating:** Update every unitary in every layer according to $U^l_j \\rightarrow e^{i \\, \\epsilon \\, K^l_j} \\ U^l_j$\n",
    "\n",
    "**Termination: Repeat Steps 1 and 2 $\\text{trainingRounds}$ times**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def qnnTraining(qnnArch, initialUnitaries, trainingData, lda, ep, trainingRounds, alert=0):\n",
    "    \n",
    "    ### FEEDFORWARD    \n",
    "    #Feedforward for given unitaries\n",
    "    s = 0\n",
    "    currentUnitaries = initialUnitaries\n",
    "    storedStates = feedforward(qnnArch, currentUnitaries, trainingData)\n",
    "\n",
    "    #Cost calculation for given unitaries\n",
    "    outputStates = []\n",
    "    for k in range(len(storedStates)):\n",
    "        outputStates.append(storedStates[k][-1])\n",
    "    plotlist = [[s], [costFunction(trainingData, outputStates)]]\n",
    "    \n",
    "    #Optional\n",
    "    runtime = time()\n",
    "    \n",
    "    #Training of the Quantum Neural Network\n",
    "    for k in range(trainingRounds):\n",
    "        if alert>0 and k%alert==0: print(\"In training round \"+str(k))\n",
    "        \n",
    "        ### UPDATING\n",
    "        newUnitaries = unitariesCopy(currentUnitaries)\n",
    "        \n",
    "        #Loop over layers:\n",
    "        for l in range(1, len(qnnArch)):\n",
    "            numInputQubits = qnnArch[l-1]\n",
    "            numOutputQubits = qnnArch[l]\n",
    "            \n",
    "            #Loop over perceptrons\n",
    "            for j in range(numOutputQubits):\n",
    "                newUnitaries[l][j] = (makeUpdateMatrixTensored(qnnArch,currentUnitaries,trainingData,storedStates,lda,ep,l,j)* currentUnitaries[l][j])\n",
    "        \n",
    "        ### FEEDFORWARD\n",
    "        #Feedforward for given unitaries\n",
    "        s = s + ep\n",
    "        currentUnitaries = newUnitaries\n",
    "        storedStates = feedforward(qnnArch, currentUnitaries, trainingData)\n",
    "        \n",
    "        #Cost calculation for given unitaries\n",
    "        outputStates = []\n",
    "        for m in range(len(storedStates)):\n",
    "            outputStates.append(storedStates[m][-1])\n",
    "        plotlist[0].append(s)\n",
    "        plotlist[1].append(costFunction(trainingData, outputStates))\n",
    "    \n",
    "    #Optional\n",
    "    runtime = time() - runtime\n",
    "    print(\"Trained \"+str(trainingRounds)+\" rounds for a \"+str(qnnArch)+\" network and \"+str(len(trainingData))+\" training pairs in \"+str(round(runtime, 2))+\" seconds\")\n",
    "    \n",
    "    #Return\n",
    "    return [plotlist, currentUnitaries]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5 Helper-functions for Tests and Applications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\text{boundRand}$ gives a formula for the estimate of the cost function, where $D$ is the dimension of the qudit that the network unitary is acting on and $N$ is the number of available training pairs of which the first $n < D$ are chosen for training:\n",
    "\n",
    "$$C \\sim \\frac{n}{N} + \\frac{N - n}{N \\cdot D \\cdot (D+1)} \\big(D + \\text{min}\\{n^2 + 1,\\, D^2\\} \\big)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def boundRand(D, N, n):\n",
    "    return (n/N) + (N-n)/(N*D*(D+1)) * (D + min(n**2+1, D**2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\text{subsetTrainingAvg}$ is given the parameters $\\text{qnnArch},\\, \\text{initialUnitaries},\\, \\text{trainingData},\\, \\lambda,\\, \\epsilon$ and $\\text{trainingRounds}$ just like $\\text{qnnTraining}$. Additionally it recieves  $n, \\text{iterations} \\in \\mathbb{N}$.\n",
    "\n",
    "$\\text{subsetTrainingAvg}$ then does the following $\\text{iterations}$-times:\n",
    "\n",
    "- Choose a subset of $\\text{trainingData}$ of size $n$\n",
    "\n",
    "- Train the given network with said subset\n",
    "\n",
    "- Calculate the cost function of the resulting network over the whole set of $\\text{trainingData}$ and store it\n",
    "\n",
    "$\\text{subsetTrainingAvg}$ ultimately returns the cost averaged over all $\\text{iterations}$ results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def subsetTrainingAvg(qnnArch, initialUnitaries, trainingData, lda, ep, trainingRounds, iterations, n, alertIt=0):\n",
    "    costpoints = []\n",
    "    \n",
    "    for i in range(iterations):\n",
    "        if alertIt>0 and i%alertIt==0: print(\"n=\"+str(n)+\", i=\"+str(i))\n",
    "        \n",
    "        #Prepare subset for training\n",
    "        trainingSubset = sample(trainingData, n)\n",
    "        \n",
    "        #Train with the subset\n",
    "        learnedUnitaries = qnnTraining(qnnArch, initialUnitaries, trainingSubset, lda, ep, trainingRounds)[1]\n",
    "        storedStates = feedforward(qnnArch, learnedUnitaries, trainingData)\n",
    "        outputStates = []\n",
    "        for k in range(len(storedStates)):\n",
    "            outputStates.append(storedStates[k][-1])\n",
    "        \n",
    "        #Calculate cost with all training data\n",
    "        costpoints.append(costFunction(trainingData, outputStates))\n",
    "    \n",
    "    return sum(costpoints)/len(costpoints)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\text{noisyDataTraining}$ is given the parameters $\\text{qnnArch},\\, \\text{initialUnitaries},\\, \\lambda,\\, \\epsilon$ and $\\text{trainingRounds}$ just like $\\text{qnnTraining}$. Additionally it recieves  $\\text{numData}, \\text{stepSize} \\in \\mathbb{N}$ and $\\text{trainingData}, \\text{noisyData}$.\n",
    "\n",
    "$\\text{noisyDataTraining}$ then does the following ***for i=0; i<numData; i+=stepSize:***\n",
    "\n",
    "- Create a list $\\text{testData}$ by combining a random sample of $\\text{trainingData}$ of size $\\text{numData}-i$ and a random sample of $\\text{noisyData}$ of size $i$\n",
    "\n",
    "- Train the given network with said $\\text{testData}$\n",
    "\n",
    "- Calculate the cost function of the resulting network over the whole set of $\\text{trainingData}$ and store it\n",
    "\n",
    "$\\text{noisyDataTraining}$ ultimately returns the list of costs for all $i$ like\n",
    "\n",
    "\\begin{align}\n",
    "\\big[ & [0, \\dots, i, \\dots]\\\\\n",
    "& [\\text{Cost}_0, \\dots, \\text{Cost}_i, \\dots] \\big]\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def noisyDataTraining(qnnArch, initialUnitaries, trainingData, noisyData, lda, ep, trainingRounds, numData, stepSize, alertP=0):\n",
    "    noisyDataPlot = [[], []]\n",
    "    \n",
    "    i = 0\n",
    "    while i <= numData:\n",
    "        if alertP>0: print(\"Currently at \"+str(i/numData)+\"% noisy data.\")\n",
    "        \n",
    "        #Prepare mixed data for traing\n",
    "        testData1 = sample(trainingData, numData - i)\n",
    "        testData2 = sample(noisyData, i)\n",
    "        if i==0: testData = testData1\n",
    "        elif i==numData: testData = testData2\n",
    "        else: testData = testData1 + testData2\n",
    "        \n",
    "        #Train with the mixed data\n",
    "        learnedUnitaries = qnnTraining(qnnArch, initialUnitaries, testData, lda, ep, trainingRounds)[1]\n",
    "        storedStates = feedforward(qnnArch, learnedUnitaries, trainingData)\n",
    "        outputStates = []\n",
    "        for k in range(len(storedStates)):\n",
    "            outputStates.append(storedStates[k][-1])\n",
    "        \n",
    "        #Calculate cost with the real training data\n",
    "        noisyDataPlot[0].append(i)\n",
    "        noisyDataPlot[1].append(costFunction(trainingData, outputStates))\n",
    "        \n",
    "        i += stepSize\n",
    "    \n",
    "    return noisyDataPlot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Tests and Applications of the QNN-Code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 Elementary tests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training a 1-2-1 network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trained 500 rounds for a [1, 2, 1] network and 10 training pairs in 45.66 seconds\n",
      "Exceeds cost of 0.95 at training step 20\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAbDklEQVR4nO3df5BdZZ3n8fcn3enO71+kCZIACRKBCEPEbHAHBwEVgsqyUqvCjOsM6ka2wHJmdplBZ3endmemdIuaKd2VMsu4qUihsNZKluhkBBd1cC2ENBISAgS6EiBNQ7qTkKR/d9/b3/3jnk5OmtvJ7eSenOTez6uqq+95zjnd39MU55Pnec4PRQRmZmZjTcq7ADMzOzU5IMzMrCwHhJmZleWAMDOzshwQZmZWVmPeBVTT/PnzY/HixXmXYWZ22njmmWf2RERLuXU1FRCLFy+mtbU17zLMzE4bkl4bb52HmMzMrCwHhJmZleWAMDOzshwQZmZWlgPCzMzKyiwgJK2V1Cnp+XHWS9J/k9QmaYuky1PrVknanqy7O6sazcxsfFn2INYBq46y/gZgafK1GvgOgKQG4N5k/TLgVknLMqzTzMzKyOw+iIh4QtLio2xyE3B/lJ43/htJcyS9C1gMtEXEDgBJDyXbvpBVraeL4kjQN1Sgb6hIz2CBvsEivUMFhosjFIrBcHGE4kgwPBIURttGSm3FkSACAhh9xHtpOd1+eJlku/HWnYpO4dI4pf9wdtqb1tzI7R96d9V/bp43yi0EdqWW25O2cu1XjPdDJK2m1APh3HPPrX6VJ1FX9yCvdHazc08vO7t6efPgAF3dg+zpGWRP9yAHBwp5l2gnQMq7AqtV82c011xAlPvfJY7SXlZE3AfcB7BixYrT6p9p+/uG+PlLnfxyexe/ff1t2t/uP7RuyuRJnD17KvNnNnPxWbOYf0ETc6c3MaO5kWlNjUxvbmBaUyPTmhpoapxE4yQxuWESDZPE5AbROGkSjQ2H2xokJBACkXwGScn30rrRk1h6+R3b+UxnVhfyDIh24JzU8iKgA2gap70mFEeCX27v5P4nX+P/te2hOBK0zGxm5eJ5/NHvLuais2Zxfst0zpo1hUmTfCI2s/zkGRAbgDuTOYYrgAMR8aakLmCppCXAG8AtwO/nWGfV/NPLXXx944u89FY3C2Y186Wrzuf6957FpQtnOwzM7JSTWUBIehC4GpgvqR34S2AyQESsATYCHwPagD7gtmRdQdKdwKNAA7A2IrZlVefJcHBgmL9Y/zw/fq6Dc+dN41u3LOdjl76LyQ2+DcXMTl1ZXsV06zHWB3DHOOs2UgqQ094ru7u5bd0m3jwwwJ9+9D186UPn09zYkHdZZmbHVFOP+z7VbG0/wOfWPkVjwyR++KV/zvvPm5t3SWZmFXNAZOTl3d38/nd/w6wpk/n+F69g8fzpeZdkZjYhDogM7Osd4vPrNjFlcgP/60sfYNHcaXmXZGY2YZ4lrbKI4O4fbWH3wQH+/nMrHA5mdtpyQFTZw799g8de2M1d11/I8nPm5F2Omdlxc0BU0cGBYb7+jy9y+blz+OIHz8+7HDOzE+I5iCr674+/wt7eIdbdttI3vpnZac89iCrp7B7g/idf4+b3LeKShbPzLsfM7IQ5IKrku7/ayXBxhC9fe0HepZiZVYUDogq6B4Z54DevceNlZ/t+BzOrGQ6IKlj/7Bv0DRW57coleZdiZlY1DogTFBE88JvXuHThbC5b5LkHM6sdDogT9Fz7AV7e3cMfXHGuX6RjZjXFAXGCfvxcB00Nk7jh0nflXYqZWVU5IE5AcST4yZYOrr6whdlTJ+ddjplZVTkgTkDrq/vYfXCQGy87O+9SzMyqzgFxAh5/qZOmhklce9GZeZdiZlZ1DogT8POXOrni/HlMb/YTS8ys9jggjtOufX20dfZw9YXuPZhZbXJAHKdfbu8E4JoLW3KuxMwsGw6I4/Trtr0smjuVJX60hpnVKAfEcYgInn51H1csOcM3x5lZzco0ICStkrRdUpuku8usnytpvaQtkp6WdElq3auStkraLKk1yzon6pXOHvb1DnHF+fPyLsXMLDOZXX4jqQG4F/go0A5skrQhIl5IbfY1YHNEfFLSRcn2H06tvyYi9mRV4/F6auc+AK5Y4oAws9qVZQ9iJdAWETsiYgh4CLhpzDbLgMcBIuIlYLGkBRnWVBVP7djLWbOmcO68aXmXYmaWmSwDYiGwK7XcnrSlPQfcDCBpJXAesChZF8Bjkp6RtDrDOifs2df38/7Fcz3/YGY1LcuAKHf2jDHL3wDmStoMfBl4Figk666MiMuBG4A7JF1V9pdIqyW1Smrt6uqqUunj29MzyBv7+1m+aE7mv8vMLE9ZBkQ7cE5qeRHQkd4gIg5GxG0RsRz4HNAC7EzWdSTfO4H1lIas3iEi7ouIFRGxoqUl+3sStrTvB+BSv/vBzGpclgGxCVgqaYmkJuAWYEN6A0lzknUAXwSeiIiDkqZLmplsMx24Dng+w1ortqX9ABJcstABYWa1LbOrmCKiIOlO4FGgAVgbEdsk3Z6sXwNcDNwvqQi8AHwh2X0BsD4Z428EfhARP82q1onY0n6AC1pmMMPPXzKzGpfpWS4iNgIbx7StSX1+ElhaZr8dwGVZ1nY8IoIt7Qe46j3z8y7FzCxzvpN6Arp6BtnTM8glZ3t4ycxqnwNiAra/1Q3ARWfNzLkSM7PsOSAmYDQgLnRAmFkdcEBMwPa3upk/o4kzZjTnXYqZWeYcEBPw8u5u9x7MrG44ICo0MhK8vLuH9yxwQJhZfXBAVGjX2330Dxc9QW1mdcMBUaG2zh4ALjjTAWFm9cEBUaGde3oBON+vGDWzOuGAqNDOPb3MmTaZudObjr2xmVkNcEBUaOeeXhaf4d6DmdUPB0SFXt3T6+ElM6srDogK9A8V6TgwwGIHhJnVEQdEBV7dW5qgXuKAMLM64oCowOgVTA4IM6snDogKvLa3D8BDTGZWVxwQFWh/u4850yb7LXJmVlccEBVof7ufRXOn5l2GmdlJ5YCowBv7+1k0Z1reZZiZnVQOiGOICNrf7mOhexBmVmccEMewr3eIgeERDzGZWd1xQBxD+9v9ACyc44Aws/rigDiG0YBYNNdzEGZWXzINCEmrJG2X1Cbp7jLr50paL2mLpKclXVLpvifLG/tL90B4DsLM6k1mASGpAbgXuAFYBtwqadmYzb4GbI6I3wE+B3xrAvueFO1v9zNzSiOzp07O49ebmeUmyx7ESqAtInZExBDwEHDTmG2WAY8DRMRLwGJJCyrc96To2D/g+Qczq0tZBsRCYFdquT1pS3sOuBlA0krgPGBRhfuS7LdaUquk1q6uriqVflhn9wALZk2p+s81MzvVZRkQKtMWY5a/AcyVtBn4MvAsUKhw31JjxH0RsSIiVrS0tJxIvWW9dWCABbOaq/5zzcxOdVk+XKgdOCe1vAjoSG8QEQeB2wAkCdiZfE071r4nQ6E4wp6eQfcgzKwuZdmD2AQslbREUhNwC7AhvYGkOck6gC8CTyShccx9T4a9vUOMBJzpgDCzOpRZDyIiCpLuBB4FGoC1EbFN0u3J+jXAxcD9korAC8AXjrZvVrWOZ/fBAQDOckCYWR3K9PnVEbER2DimbU3q85PA0kr3PdneOlAKCM9BmFk98p3UR7G7exDAcxBmVpccEEfReXCASYIzpjcde2MzsxrjgDiK3QcHmD+jmcYG/5nMrP74zHcUuw8OctZsDy+ZWX1yQBzF7oMDnDnTAWFm9ckBcRRd3YO0zPQVTGZWnxwQ4yiOBG/3DTF/hieozaw+OSDGsb+vdBf1PF/BZGZ1ygExjn29QwCcMcNDTGZWnxwQ49jTkwSEexBmVqccEOM43INwQJhZfXJAjGNvb+kxG56DMLN65YAYx95kiGneNAeEmdUnB8Q49vYOMnfaZD9mw8zqls9+49jXO+ThJTOra8d8H4Skmyv4OQPJ+xtqxp6eIV/iamZ1rZIXBv098Aigo2xzFTm/3Kfa9vUOsfTMGXmXYWaWm0oC4h8j4vNH20DSA1Wq55Sxt2eQD5w/L+8yzMxyc8w5iIj4bDW2OZ0UiiPs7x9m3nQPMZlZ/ap4klrSpyTNTD7/B0kPS7o8u9Lys79/mAjfRW1m9W0iVzH9x4jolvRB4Hrge8B3sikrX/v7hgGYM21yzpWYmeVnIgFRTL5/HPhORDwC1OQ/sQ/0lwJi9lQHhJnVr4kExBuS/gfwaWCjpOZj7S9plaTtktok3V1m/WxJP5b0nKRtkm5LrXtV0lZJmyW1TqDOE3agv3QXtQPCzOrZRALi08CjwKqI2A/MA+4ab2NJDcC9wA3AMuBWScvGbHYH8EJEXAZcDfytpHSv5JqIWB4RKyZQ5wlzD8LMrLLLXAGIiD7g4dTym8CbR9llJdAWETsAJD0E3AS8kP6xwExJAmYA+4BCxdVn5ECfA8LM7Jg9CEm/Pc5tFgK7UsvtSVvat4GLgQ5gK/CViBhJ1gXwmKRnJK0+yu9eLalVUmtXV9exSq3IfvcgzMwq6kFcLGnLUdYLmD1O+1gxZvl6YDNwLfBu4GeSfhURB4ErI6JD0plJ+0sR8cQ7fmDEfcB9ACtWrBj784/Lgf5hZjQ3+kF9ZlbXKgmIiyrYplimrR04J7W8iFJPIe024BsREUCbpJ3J73s6IjoAIqJT0npKQ1bvCIgsHOgfdu/BzOpeJXdSvxYRrwF/Pfq5TFt7mV03AUslLUkmnm8BNozZ5nXgwwCSFgAXAjskTU/dlDcduA54/ngPcqIO9g8zywFhZnWu4klq4L3pheQqpfePt3FEFCTdSenKpwZgbURsk3R7sn4N8FfAOklbKQ1J/XlE7JF0PrC+NHdNI/CDiPjpBGo9Ifv7hpnjgDCzOlfJ476/CnwNmCrp4GgzMEQy9j+e5BHgG8e0rUl97qDUOxi73w7gsmPVlpUD/cO8u8VPcjWz+lbJENPXI2ImcE9EzEq+ZkbEGRHx1ZNQ40nnOQgzs4ndKPeTZD4ASZ+V9HeSzsuorlzt7x9mtp/DZGZ1biIB8R2gT9JlwJ8BrwH3Z1JVjgaGiwwVRtyDMLO6N5GAKCSXo94EfCsivgXMzKas/PgxG2ZmJRO5iqk7mbD+18DvJVcx1dxZ1AFhZlYykR7EZ4BB4PMR8Ralx2bck0lVOdrv5zCZmQETCIgkFL4PzJb0CWAgImpuDsI9CDOzkom8cvTTwNPApyg9+vspSf8qq8Ly0jNYCoiZUyYy+mZmVnsmchb8C+CfRUQngKQW4P8C/zuLwvLSM1h6rNQMB4SZ1bmJzEFMGg2HxN4J7n9a6BkovY5iRrMDwszq20TOgj+V9CjwYLL8GcY8RqMW9A4WmCSYOrkh71LMzHJVybOYLgAWRMRdkm4GPkjpWUxPUpq0rik9gwVmNDeSPCjQzKxuVTJE9E2gGyAiHo6IP42IP6HUe/hmlsXloXug4OElMzMqC4jFEfGON8pFRCuwuOoV5ax3sOAJajMzKguIKUdZN7VahZwqRoeYzMzqXSUBsUnSvxnbKOkLwDPVLylfPYMFpjsgzMwquorpjym93e0POBwIK4Am4JNZFZaXnsECZ885WqfJzKw+HDMgImI38LuSrgEuSZr/ISJ+nmllOekdLDC9yT0IM7OKz4QR8QvgFxnWckroGfAktZkZ1OCd0CciIugZKjDTcxBmZg6ItL6hIhF4ktrMDAfEEXoGk+cweYjJzMwBkXYoINyDMDPLNiAkrZK0XVKbpLvLrJ8t6ceSnpO0TdJtle6bBT/J1czssMwCInln9b3ADcAy4FZJy8ZsdgfwQkRcBlwN/K2kpgr3rTr3IMzMDsuyB7ESaIuIHRExBDwE3DRmmwBmqvTo1BnAPqBQ4b5V5zkIM7PDsgyIhcCu1HJ70pb2beBioAPYCnwlIkYq3BcASasltUpq7erqOqGCPcRkZnZYlgFR7oUKMWb5emAzcDawHPi2pFkV7ltqjLgvIlZExIqWlpYTqZfeIQeEmdmoLAOiHTgntbyIUk8h7Tbg4ShpA3YCF1W4b9V1D3iIycxsVJYBsQlYKmmJpCbgFmDDmG1eBz4MIGkBcCGwo8J9q653sEDjJNHc6NeNmpll9k/liChIuhN4FGgA1kbENkm3J+vXAH8FrJO0ldKw0p9HxB6AcvtmVeuovqEi05ocDmZmkGFAAETERkqvJk23rUl97gCuq3TfrA0MF5nqgDAzA3wn9RFKPQjPP5iZgQPiCP3DRaZMdg/CzAwcEEfo9xyEmdkhDoiUvqECU92DMDMDHBBH6B8e8SS1mVnCAZHS7x6EmdkhDoiU/mHPQZiZjXJApPQN+SomM7NRDoiUAfcgzMwOcUAkhosjDBfDcxBmZgkHRKJ/uAjgq5jMzBIOiET/kAPCzCzNAZEYDQjPQZiZlTggEn2jPQjPQZiZAQ6IQw7PQfhprmZm4IA4pN89CDOzIzggEqM9CM9BmJmVOCASfUMFAN9JbWaWcEAkBtyDMDM7ggMi4auYzMyO5IBI+E5qM7MjOSAS/UNFJGhu9J/EzAwyDghJqyRtl9Qm6e4y6++StDn5el5SUdK8ZN2rkrYm61qzrBNKATF1cgOSsv5VZmanhczuCpPUANwLfBRoBzZJ2hARL4xuExH3APck298I/ElE7Ev9mGsiYk9WNab1+VHfZmZHyLIHsRJoi4gdETEEPATcdJTtbwUezLCeoxrwy4LMzI6QZUAsBHalltuTtneQNA1YBfwo1RzAY5KekbR6vF8iabWkVkmtXV1dx11s35B7EGZmaVkGRLnB/Bhn2xuBX48ZXroyIi4HbgDukHRVuR0j4r6IWBERK1paWo672P7hoi9xNTNLyTIg2oFzUsuLgI5xtr2FMcNLEdGRfO8E1lMasspM/3CRZgeEmdkhWQbEJmCppCWSmiiFwIaxG0maDXwIeCTVNl3SzNHPwHXA8xnWymBhxHMQZmYpmV3FFBEFSXcCjwINwNqI2Cbp9mT9mmTTTwKPRURvavcFwPrkktNG4AcR8dOsagUYHC7SPLM5y19hZnZayfTlBxGxEdg4pm3NmOV1wLoxbTuAy7KsbayhwohvkjMzS/EZMTFYGKG50UNMZmajHBCJwUKRKZP95zAzG+UzYmJw2D0IM7M0B0RisDBCs3sQZmaH+IwIjIwEQ0VPUpuZpfmMCAwVRwA8xGRmluKAoDT/AH4XhJlZms+IlK5gAjwHYWaW4jMipQlq8BCTmVmaA4JUD8JDTGZmh/iMCAx4DsLM7B18RiQ1xOSnuZqZHeKAwENMZmbl+IxIepLafw4zs1E+I1J6FwT4KiYzszQHBOk5CP85zMxG+YyI76Q2MyvHZ0TSk9QeYjIzG+WAwENMZmbl+IyIr2IyMyvHZ0QOX8XU1OA/h5nZKJ8RSd4m1zgJSXmXYmZ2ysg0ICStkrRdUpuku8usv0vS5uTreUlFSfMq2beaRgPCzMwOy+ysKKkBuBe4AVgG3CppWXqbiLgnIpZHxHLgq8A/RcS+SvatpsFC0c9hMjMbI8t/Nq8E2iJiR0QMAQ8BNx1l+1uBB49z3xMyOOwehJnZWFmeFRcCu1LL7UnbO0iaBqwCfjTRfathsDDCFPcgzMyOkGVAlJvxjXG2vRH4dUTsm+i+klZLapXU2tXVdRxlJkNM7kGYmR0hy7NiO3BOankR0DHOtrdweHhpQvtGxH0RsSIiVrS0tBxXoZ6kNjN7pyzPipuApZKWSGqiFAIbxm4kaTbwIeCRie5bLaU5CA8xmZmlNWb1gyOiIOlO4FGgAVgbEdsk3Z6sX5Ns+kngsYjoPda+WdU6WCgyd3pTVj/ezOy0lFlAAETERmDjmLY1Y5bXAesq2TcrHmIyM3snnxUZDQgPMZmZpTkgKD2LyT0IM7Mj+axI0oPwo77NzI7gsyIeYjIzK8cBAXzk4jN579mz8i7DzOyUkulVTKeLb97yvrxLMDM75bgHYWZmZTkgzMysLAeEmZmV5YAwM7OyHBBmZlaWA8LMzMpyQJiZWVkOCDMzK0sR470F9PQjqQt47Th3nw/sqWI5pwMfc+2rt+MFH/NEnRcRZV/HWVMBcSIktUbEirzrOJl8zLWv3o4XfMzV5CEmMzMrywFhZmZlOSAOuy/vAnLgY6599Xa84GOuGs9BmJlZWe5BmJlZWQ4IMzMrq+4DQtIqSdsltUm6O+96siBpraROSc+n2uZJ+pmkV5Lvc/OssdoknSPpF5JelLRN0leS9po9bklTJD0t6bnkmP9z0l6zxwwgqUHSs5J+kizX9PECSHpV0lZJmyW1Jm1VP+66DghJDcC9wA3AMuBWScvyrSoT64BVY9ruBh6PiKXA48lyLSkA/y4iLgY+ANyR/Let5eMeBK6NiMuA5cAqSR+gto8Z4CvAi6nlWj/eUddExPLU/Q9VP+66DghgJdAWETsiYgh4CLgp55qqLiKeAPaNab4J+F7y+XvAvzypRWUsIt6MiN8mn7spnUAWUsPHHSU9yeLk5Cuo4WOWtAj4OPDdVHPNHu8xVP246z0gFgK7UsvtSVs9WBARb0LpZAqcmXM9mZG0GHgf8BQ1ftzJcMtmoBP4WUTU+jF/E/gzYCTVVsvHOyqAxyQ9I2l10lb142480R9wmlOZNl/3W0MkzQB+BPxxRByUyv0nrx0RUQSWS5oDrJd0Sd41ZUXSJ4DOiHhG0tV513OSXRkRHZLOBH4m6aUsfkm99yDagXNSy4uAjpxqOdl2S3oXQPK9M+d6qk7SZErh8P2IeDhprvnjBoiI/cAvKc091eoxXwn8C0mvUhoevlbSA9Tu8R4SER3J905gPaXh8qofd70HxCZgqaQlkpqAW4ANOdd0smwA/jD5/IfAIznWUnUqdRX+J/BiRPxdalXNHreklqTngKSpwEeAl6jRY46Ir0bEoohYTOn/3Z9HxGep0eMdJWm6pJmjn4HrgOfJ4Ljr/k5qSR+jNI7ZAKyNiL/JuaSqk/QgcDWlRwLvBv4S+D/AD4FzgdeBT0XE2Ins05akDwK/ArZyeHz6a5TmIWryuCX9DqXJyQZK//j7YUT8F0lnUKPHPCoZYvr3EfGJWj9eSedT6jVAaZrgBxHxN1kcd90HhJmZlVfvQ0xmZjYOB4SZmZXlgDAzs7IcEGZmVpYDwszMynJAmJlZWQ4IMzMrywFhlqHkrtd/SN7R8Lykz+Rdk1ml6v1hfWZZWwV0RMTHASTNzrkes4q5B2GWra3ARyT9V0m/FxEH8i7IrFIOCLMMRcTLwPspBcXXJf2nnEsyq5iHmMwyJOlsYF9EPCCpB/ijnEsyq5gDwixblwL3SBoBhoF/m3M9ZhXz01zNzKwsz0GYmVlZDggzMyvLAWFmZmU5IMzMrCwHhJmZleWAMDOzshwQZmZW1v8HcchC/Jok3v8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "network121 = randomNetwork([1,2,1], 10)\n",
    "plotlist121 = qnnTraining(network121[0], network121[1], network121[2], 1, 0.1, 500)[0]\n",
    "\n",
    "for i in range(len(plotlist121[1])):\n",
    "    if plotlist121[1][i] >= 0.95:\n",
    "        print(\"Exceeds cost of 0.95 at training step \"+str(i))\n",
    "        break\n",
    "\n",
    "plt.plot(plotlist121[0], plotlist121[1])\n",
    "plt.xlabel(\"s\")\n",
    "plt.ylabel(\"Cost[s]\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training a 2-3-2 network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trained 500 rounds for a [2, 3, 2] network and 10 training pairs in 99.63 seconds\n",
      "Exceeds cost of 0.95 at training step 54\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAdAUlEQVR4nO3de5RdZZ3m8e9Tt9zJtQiQOxiCUS5CGXtsFWhbDUqb1kEFL3jtLFzSY09Pq+jM2KvH5bIdV7u0R9pMpmXA9sJyjaARo8h4aZxWIQHDJUBCmQCpVFKpSiWVqtS96jd/nJ1wOFQqp1Jn16k6+/msVavO3vs9u3474nnO++69362IwMzMsquq3AWYmVl5OQjMzDLOQWBmlnEOAjOzjHMQmJllXE25CxirRYsWxcqVK8tdhpnZlPLQQw+1RUT9SNumXBCsXLmS7du3l7sMM7MpRdKzp9rmoSEzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8u41IJA0m2SDkl6/BTbJekfJTVKelTS5WnVYmZmp5Zmj+B2YP0o268BVic/G4Gvp1iLmZmdQmr3EUTE/ZJWjtJkA/DNyM2D/TtJ8ySdGxEH0qrJcoaHg77BYbr7B+kZGKJ3YIie/mH6h4YZGg4Gh4cZHIrkdTA4NJz7nbd+OCAIIiAiCHjB6+HkNSfWk7yn8H3ByfeOR+6vjnMf466hBDwtvI2iYeUCXnfhiPeEjUs5byhbAuzLW25K1r0oCCRtJNdrYPny5RNS3FTSPzjMwY5e9h/t4UBHD81He2jr6udodz9HewY40j1AR3c/HT0DyQf/cLlLtlFI5a7AJqubrryg4oJgpP/cR/w6FBGbgc0ADQ0Nmf7KdOR4P9ueaWdn8zF2t3Syq6WTZ9qOM1zwrzJneg3zZtYyf2Ydc2fUsmLBTObOqGVmXTXTa6uZUVfNjNrkJ1lXWy1qqqqoqRY1VaKmuoqaKlFdJWqrRXXV88tVElLuQ0skrwFJVCXrOLmd59tz6veN9/OvFB+g8qewZVA5g6AJWJa3vBRoLlMtk9bQcLDtmXbu3XmQ3/7hME8d7ARyH3orF85izeI5vOXic1m2YCZL5s3gvHkzOHfudKbXVpe5cjObKsoZBFuAmyXdCbwK6PD5gec9e/g4337gOe56uIm2rn7qaqp41aoFXHvJubzq/IVcvGSuP+zNrCRSCwJJ3wWuAhZJagL+FqgFiIhNwFbgzUAj0A18MK1appKnDh7jH362m/ueaKG6SrzhpYv5s0vP46o19cyaNuXmCDSzKSDNq4ZuOM32AD6W1t+falo7+/jC1ie5e8d+Zk+r4T+8fjXvXrecc+ZOL3dpZlbh/BVzEvjhjv387ZaddPcPsfF15/PRKy9g3sy6cpdlZhnhICijvsEh/u5HT/CdB57j8uXz+O/XXcpLzp5d7rLMLGMcBGXS0TPAR+7YxrZnjnDTlRfwiTetobrKly6a2cRzEJTB4a4+brztQXa3dPKPN7yCt156XrlLMrMMcxBMsM7eAd73jQf5Q2sXm29s4Oo1Z5e7JDPLOAfBBOofHOaj33qYXS2dfOP9DVzlEDCzScDPI5hAn7vnCf5fYxt///aLHQJmNmk4CCbIPY828y+/e5a/eO0q3tGw7PRvMDObIA6CCbCvvZtbvv8Yr1g+j0+uv6jc5ZiZvYCDIGURwWfufgyA/3HDK6it9j+5mU0u/lRK2d2/38+vn27jU+vXsHT+zHKXY2b2Ig6CFB3t7udz9zzB5cvn8Z5XrSh3OWZmI3IQpOjWXzZytGeAz7/tYqp817CZTVIOgpTsa+/mjt88y3WXL+Wl555V7nLMzE7JQZCSL9+3Gwn++o0XlrsUM7NROQhSsLftOD/YsZ8PvHol586dUe5yzMxG5SBIweb791BbXcVHXnt+uUsxMzutVINA0npJuyQ1SrplhO3zJd0t6VFJD0p6eZr1TIRDnb18/+Em3nHFUurnTCt3OWZmp5VaEEiqBm4FrgHWAjdIWlvQ7DPAjoi4BLgR+Gpa9UyU//1vzzA4NMzG17k3YGZTQ5o9gnVAY0TsiYh+4E5gQ0GbtcDPASLiKWClpMUp1pSq3oEhvvvgc7zpZeewYuGscpdjZlaUNINgCbAvb7kpWZfvEeDtAJLWASuApYU7krRR0nZJ21tbW1Mqd/zu3XmQo90DvnnMzKaUNINgpDuoomD574H5knYAfwn8Hhh80ZsiNkdEQ0Q01NfXl77SEvnOA8+xfMFMXn3BwnKXYmZWtDQfTNME5M+3vBRozm8QEceADwJIErA3+Zly/tDaxQN72/nk+jW+i9jMppQ0ewTbgNWSVkmqA64HtuQ3kDQv2QbwEeD+JBymnO9t20dNlbjuiheNbJmZTWqp9QgiYlDSzcC9QDVwW0TslHRTsn0T8FLgm5KGgCeAD6dVT5qGh4MfPdLM6y6s5+w508tdjpnZmKT6zOKI2ApsLVi3Ke/1b4HVadYwEX6/7wjNHb38zZvWlLsUM7Mx853FJXDPoweoq6niDWun7JWvZpZhDoJxGh4Otj52gKsurGfO9Npyl2NmNmYOgnHa9kw7Lcf6uPbS88pdipnZGXEQjNN9T7RQV13F6y86u9ylmJmdEQfBOP1qdyuvOn8Bs6alet7dzCw1DoJx2NfeTeOhLq5a496AmU1dDoJx+NXu3LxHV6+ZvNNemJmdjoNgHH711CGWL5jJqkWeadTMpi4HwRnqHRjiN384zNVr6slNk2RmNjU5CM7QQ88eoWdgiCs9LGRmU5yD4Aw9sLedKsG6VZ5y2symNgfBGXpw72FevmQus33ZqJlNcQ6CM9A3OMTvnzvKupULyl2Kmdm4OQjOwGNNHfQNDrNulYPAzKY+B8EZeGBvOwCvdI/AzCqAg+AMPLi3nTWL5zB/Vt3pG5uZTXKpBoGk9ZJ2SWqUdMsI2+dK+pGkRyTtlPTBNOsphcGhYR569oiHhcysYqQWBJKqgVuBa4C1wA2S1hY0+xjwRERcClwF/EPeM4wnpacPddHVN8gVK+aXuxQzs5JIs0ewDmiMiD0R0Q/cCWwoaBPAHOVuzZ0NtAODKdY0bo81dQBwydK5Za7EzKw00gyCJcC+vOWmZF2+r5F7gH0z8Bjw8YgYTrGmcXt0/1HmTK9h5ULPL2RmlSHNIBhpAp4oWH4TsAM4D7gM+Jqks160I2mjpO2Stre2tpa+0jF4tKmDi5fMparK8wuZWWVIMwiagGV5y0vJffPP90HgrshpBPYCFxXuKCI2R0RDRDTU15dvbp++wSGePHCMiz0sZGYVJM0g2AaslrQqOQF8PbCloM1zwOsBJC0G1gB7UqxpXHYf7GJgKLh06bxyl2JmVjKpTZQTEYOSbgbuBaqB2yJip6Sbku2bgM8Bt0t6jNxQ0qcioi2tmsbrkaajAFy8xD0CM6scqc6YFhFbga0F6zblvW4G3phmDaX0WFMH82fWsnT+jHKXYmZWMr6zeAwe3d/BJUvn+UE0ZlZRHARF6h8c5umWTtae96KLmszMpjQHQZH2th1ncDi46Jw55S7FzKykHARFeurgMQDWOAjMrMI4CIq0u6WTmipx/qLZ5S7FzKykHARF2nWwk/PrZ1FX438yM6ss/lQr0q6WTi5c7GEhM6s8DoIidPUNsq+9xyeKzawiOQiK8HRLJ4B7BGZWkRwERdh1MBcEF53jewjMrPI4CIqwq6WTGbXVnlrCzCqSg6AIT7d0sXrxbD+DwMwqkoOgCHtau7ig3vcPmFllchCcRk//EM0dvZy/yI+mNLPK5CA4jWcOHwdgVb2DwMwqk4PgNPa0JkHgHoGZVSgHwWnsbesCYOVCB4GZVaZUg0DSekm7JDVKumWE7Z+QtCP5eVzSkKQFadY0VnvajnPOWdOZNS3Vh7mZmZVNakEgqRq4FbgGWAvcIGltfpuI+FJEXBYRlwGfBv41ItrTqulM7G077mEhM6toafYI1gGNEbEnIvqBO4ENo7S/AfhuivWckb1txznfJ4rNrIKlGQRLgH15y03JuheRNBNYD3z/FNs3StouaXtra2vJCz2V9uP9HO0ecI/AzCpamkEw0m24cYq2fwb826mGhSJic0Q0RERDfX19yQo8nRMnit0jMLNKlmYQNAHL8paXAs2naHs9k3BY6PlLR31XsZlVrjSDYBuwWtIqSXXkPuy3FDaSNBe4EvhhirWckb1tx6mpkiebM7OKlto1kRExKOlm4F6gGrgtInZKuinZvilp+jbgZxFxPK1aztRz7d0smT+D2mrfbmFmlSvVi+MjYiuwtWDdpoLl24Hb06zjTO070uPegJlVPH/VHcX+I90smz+z3GWYmaXqtD0CSW8vYj+9ybf/itHdP0hbV797BGZW8YoZGvpf5E7kjvZUltdRMAQ01e0/0gPAsgXuEZhZZSsmCH4SER8arYGkb5Wonklj35FuAJZ6aMjMKtxpzxFExHtL0WaqaTrRI/DQkJlVuKJPFkt6h6Q5yev/IukuSZenV1p57WvvZlpNFfVzppW7FDOzVI3lqqH/GhGdkl4DvAm4A/h6OmWV3772HpbMn4HkB9abWWUbSxAMJb/fAnw9In4I1JW+pMmh6agvHTWzbBhLEOyX9D+BdwJbJU0b4/unlH3tPSxb4PMDZlb5xvJB/k5y00Wsj4ijwALgE6lUVWbHegfo6BnwFUNmlglFTzEREd3AXXnLB4ADaRRVbk3tJ64YchCYWeU7bY9A0sOlaDOVNJ28h8BDQ2ZW+YrpEbxU0qOjbBcwt0T1TArNR3M9giUOAjPLgGKC4KIi2gydvsnUceBYL3XVVSyYWbEXRZmZnXTaIIiIZwEk/UtEvC9/20jrKkFLRy9nnzWNqirfQ2BmlW8sVw29LH9BUjVwRWnLmRwOdPRyzlnTy12GmdmEKOZk8acldQKXSDqW/HQCh5iEj5cshZZjvZwz10FgZtlQzKRzX4iIOcCXIuKs5GdORCyMiE+P9l5J6yXtktQo6ZZTtLlK0g5JOyX96xkeR8lEBAePuUdgZtkxlqGheyTNApD0XklflrTiVI2ToaNbgWuAtcANktYWtJkH/BPw1oh4GfCOsR5AqXX0DNA7MOwegZllxliC4OtAt6RLgU8CzwLfHKX9OqAxIvZERD9wJ7ChoM27gbsi4jmAiDg0hnpScfBYL4CDwMwyYyxBMBgRQe7D/KsR8VVgzijtlwD78pabknX5LgTmS/qVpIck3TjSjiRtlLRd0vbW1tYxlDx2BzuSIPDQkJllRNFTTACdkj4NvA94bTL0UztK+5GuvYwR/v4VwOuBGcBvJf0uIna/4E0Rm4HNAA0NDYX7KKmTQeAegZllxFh6BO8C+oAPRcRBct/uvzRK+yZgWd7yUqB5hDY/jYjjEdEG3A9cOoaaSu7E0NDZcxwEZpYNRQdB8uH/bWCupGuB3ogY7RzBNmC1pFWS6oDrgS0FbX5IrndRI2km8CrgyTEdQYm1HOtl0ew66moqdoZtM7MXGMujKt8JPEjuyp53Ag9Iuu5U7SNiELiZ3NTVTwLfi4idkm6SdFPS5kngp8Cjyb7/OSIeP9ODKYUDHb6HwMyyZSznCP4z8MoTV/ZIqgf+L/B/TvWGiNgKbC1Yt6lg+UuMPsQ0oQ529HrWUTPLlLGMf1QVXN55eIzvnxJ8V7GZZc1YegQ/lXQv8N1k+V0UfNuf6noHhjjSPeBLR80sU04bBJJeAiyOiE9IejvwGnKXhv6W3MnjitGSXDG02EFgZhlSzNDOV4BOgIi4KyL+OiL+I7newFfSLG6itRzrAxwEZpYtxQTByoh40RPKImI7sLLkFZVRW1cuCOrnTCtzJWZmE6eYIBjt63FFXV7T2ukgMLPsKSYItkn6i8KVkj4MPFT6ksqnrauPKsF8P6LSzDKkmKuG/gq4W9J7eP6DvwGoA96WVmHl0NrZx8LZ06j2IyrNLEOKeWZxC/BqSVcDL09W/zgifpFqZWXQ1tVH/WwPC5lZthR9H0FE/BL4ZYq1lF1rZx+LfH7AzDKm4u4MHo+2rn73CMwscxwEiYhIegQ+UWxm2eIgSBzrGaR/aNg9AjPLHAdBotU3k5lZRjkIEidvJnOPwMwyxkGQODG9hK8aMrOsSTUIJK2XtEtSo6RbRth+laQOSTuSn8+mWc9o3CMws6way/MIxkRSNXAr8AZyD6nfJmlLRDxR0PTXEXFtWnUUq62rj9pqMXdGbblLMTObUGn2CNYBjRGxJyL6gTuBDSn+vXFp7exj4axpVHl6CTPLmDSDYAmwL2+5KVlX6N9JekTSTyS9LMV6RtXW1ecrhswsk1IbGiL3FLNCUbD8MLAiIrokvRn4AbD6RTuSNgIbAZYvX17qOoHc5aM+P2BmWZRmj6AJWJa3vBRozm8QEccioit5vRWolbSocEcRsTkiGiKiob6+PpViWzvdIzCzbEozCLYBqyWtklQHXA9syW8g6RxJSl6vS+o5nGJNIxoeDg539bPIPQIzy6DUhoYiYlDSzcC9QDVwW0TslHRTsn0TcB3wUUmDQA9wfUQUDh+lrqNngMHhcBCYWSaleY7gxHDP1oJ1m/Jefw34Wpo1FMM3k5lZlvnOYp6fZ2jRbM88ambZ4yAg9xwC8F3FZpZNDgKgrfNEj8BBYGbZ4yAgd46gpsrTS5hZNjkIyAXBwtl1nl7CzDLJQQAc7upn4SwPC5lZNjkIyPUIfOmomWWVg4DcVUO+dNTMsirzQRARnnDOzDIt80HQ2TdI/+CwLx01s8zKfBCcvIdgjoeGzCybMh8Eh4/n7ir2VUNmllWZDwLfVWxmWecg6PLQkJllW+aDoLWrHwkWzHQQmFk2ZT4I2rr6WDCzjprqzP9TmFlGZf7Tr62zz+cHzCzTUg0CSesl7ZLUKOmWUdq9UtKQpOvSrGckueklPCxkZtmVWhBIqgZuBa4B1gI3SFp7inZfJPds4wl3+LgnnDOzbEuzR7AOaIyIPRHRD9wJbBih3V8C3wcOpVjLKXloyMyyLs0gWALsy1tuStadJGkJ8DZgE6OQtFHSdknbW1tbS1ZgT/8Qx/uHPDRkZpmWZhCM9JSXKFj+CvCpiBgabUcRsTkiGiKiob6+vmQFnryHwD0CM8uwmhT33QQsy1teCjQXtGkA7pQEsAh4s6TBiPhBinWd1JoEgWceNbMsSzMItgGrJa0C9gPXA+/ObxARq068lnQ7cM9EhQB4egkzM0gxCCJiUNLN5K4GqgZui4idkm5Kto96XmAitHUlE875oTRmlmFp9giIiK3A1oJ1IwZARHwgzVpGcuIcgYPAzLIs03cWtxzrZcGsOqbVVJe7FDOzssl4EPRxth9ab2YZl+kgONTZy+Kzppe7DDOzssp0ELQc62XxWe4RmFm2ZTYIhoaDtq5+9wjMLPMyGwSHj/cxNBw+R2BmmZfZIDh0LHfp6NnuEZhZxmU2CFqO9QJ4aMjMMi/DQZDrEfhksZllXYaDoBfJ8wyZmWU2CA519rJw1jRq/dB6M8u4zH4KHuzwPQRmZpDhIGg+2st582aUuwwzs7LLcBD0sMRBYGaWzSA41jtAZ98g583zpaNmZpkMguajPQAeGjIzw0FQ5krMzMov1SCQtF7SLkmNkm4ZYfsGSY9K2iFpu6TXpFnPCfuP5u4q9jkCM7MUH1UpqRq4FXgD0ARsk7QlIp7Ia/ZzYEtEhKRLgO8BF6VV0wkHjvZQWy3qfTOZmVmqPYJ1QGNE7ImIfuBOYEN+g4joiohIFmcBwQRoPtrDOXOnU1WlifhzZmaTWppBsATYl7fclKx7AUlvk/QU8GPgQyPtSNLGZOhoe2tr67gLaz7ay3lzPSxkZgbpBsFIX7df9I0/Iu6OiIuAPwc+N9KOImJzRDREREN9ff24C9t/tMcnis3MEmkGQROwLG95KdB8qsYRcT9wgaRFKdZE3+AQBzp6WLZgZpp/xsxsykgzCLYBqyWtklQHXA9syW8g6SWSlLy+HKgDDqdYE01HehgOWOEgMDMDUrxqKCIGJd0M3AtUA7dFxE5JNyXbNwH/HrhR0gDQA7wr7+RxKp473A3AykUOAjMzSDEIACJiK7C1YN2mvNdfBL6YZg2Fnj18HIDlC2ZN5J81M5u0Mndn8TOHu5lZV82i2XXlLsXMbFLIXBA8197NioWzSE5NmJllXuaC4JnDx32i2MwsT6aCYGg4aGrvYYVPFJuZnZSpIDjQ0UP/0DArfKLYzOykTAXB04e6AHjJ2bPLXImZ2eSRqSBobMkFwWoHgZnZSZkKgt0tnSyaPY35s3zpqJnZCZkKgqcPdbk3YGZWIDNBEBE0HuriwsUOAjOzfJkJggMdvXT1DfKSxXPKXYqZ2aSSmSA4ccWQh4bMzF4oM0Ews66aN6xdzIXuEZiZvUCqs49OJq9cuYBXrlxQ7jLMzCadzPQIzMxsZA4CM7OMSzUIJK2XtEtSo6RbRtj+HkmPJj+/kXRpmvWYmdmLpRYEkqqBW4FrgLXADZLWFjTbC1wZEZcAnwM2p1WPmZmNLM0ewTqgMSL2REQ/cCewIb9BRPwmIo4ki78DlqZYj5mZjSDNIFgC7MtbbkrWncqHgZ+kWI+ZmY0gzctHR3oWZIzYULqaXBC85hTbNwIbAZYvX16q+szMjHR7BE3AsrzlpUBzYSNJlwD/DGyIiMMj7SgiNkdEQ0Q01NfXp1KsmVlWKWLEL+nj37FUA+wGXg/sB7YB746InXltlgO/AG6MiN8Uud9W4NkzLGsR0HaG752qfMzZ4GPOhvEc84qIGPGbdGpDQxExKOlm4F6gGrgtInZKuinZvgn4LLAQ+CdJAIMR0XCa/Z5xl0DS9tPtv9L4mLPBx5wNaR1zqlNMRMRWYGvBuk15rz8CfCTNGszMbHS+s9jMLOOyFgRZvGHNx5wNPuZsSOWYUztZbGZmU0PWegRmZlbAQWBmlnGZCYLTzYRaCSTdJumQpMfz1i2QdJ+kp5Pf88tZY6lJWibpl5KelLRT0seT9RV53JKmS3pQ0iPJ8f5dsr4ijzefpGpJv5d0T7Jc0ccs6RlJj0naIWl7si6VY85EEBQ5E2oluB1YX7DuFuDnEbEa+HmyXEkGgf8UES8F/gj4WPK/baUedx/wJxFxKXAZsF7SH1G5x5vv48CTectZOOarI+KyvHsHUjnmTAQBRcyEWgki4n6gvWD1BuCO5PUdwJ9PaFEpi4gDEfFw8rqT3AfFEir0uCOnK1msTX6CCj3eEyQtBd5CbjqaEyr6mE8hlWPOShCMdSbUSrI4Ig5A7kMTOLvM9aRG0krgFcADVPBxJ0MkO4BDwH0RUdHHm/gK8ElgOG9dpR9zAD+T9FAy8SakdMxZeXh90TOh2tQkaTbwfeCvIuJYMmVJRYqIIeAySfOAuyW9vNw1pUnStcChiHhI0lXlrmcC/XFENEs6G7hP0lNp/aGs9AiKmgm1QrVIOhcg+X2ozPWUnKRaciHw7Yi4K1ld8ccdEUeBX5E7L1TJx/vHwFslPUNuWPdPJH2Lyj5mIqI5+X0IuJvcEHcqx5yVINgGrJa0SlIdcD2wpcw1TZQtwPuT1+8HfljGWkpOua/+3wCejIgv522qyOOWVJ/0BJA0A/hT4Ckq9HgBIuLTEbE0IlaS+//uLyLivVTwMUuaJWnOidfAG4HHSemYM3NnsaQ3kxtnPDET6ufLXFLJSfoucBW5qWpbgL8FfgB8D1gOPAe8IyIKTyhPWZJeA/waeIznx48/Q+48QcUdd/L8jjvI/XdcBXwvIv6bpIVU4PEWSoaG/iYirq3kY5Z0PrleAOSG8L8TEZ9P65gzEwRmZjayrAwNmZnZKTgIzMwyzkFgZpZxDgIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwK4HkTtAfJ88JeFzSu8pdk1mxsjLpnFna1gPNEfEWAElzy1yPWdHcIzArjceAP5X0RUmvjYiOchdkViwHgVkJRMRu4ApygfAFSZ8tc0lmRfPQkFkJSDoPaI+Ib0nqAj5Q5pLMiuYgMCuNi4EvSRoGBoCPlrkes6J59lEzs4zzOQIzs4xzEJiZZZyDwMws4xwEZmYZ5yAwM8s4B4GZWcY5CMzMMu7/Aw2+gUnu1GKPAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "network232 = randomNetwork([2,3,2], 10)\n",
    "plotlist232 = qnnTraining(network232[0], network232[1], network232[2], 1, 0.1, 500)[0]\n",
    "\n",
    "for i in range(len(plotlist232[1])):\n",
    "    if plotlist232[1][i] >= 0.95:\n",
    "        print(\"Exceeds cost of 0.95 at training step \"+str(i))\n",
    "        break\n",
    "\n",
    "plt.plot(plotlist232[0], plotlist232[1])\n",
    "plt.xlabel(\"s\")\n",
    "plt.ylabel(\"Cost[s]\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Varying $\\lambda$ for a 2-2 network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trained 200 rounds for a [2, 2] network and 10 training pairs in 12.32 seconds\n",
      "Trained 200 rounds for a [2, 2] network and 10 training pairs in 13.62 seconds\n",
      "Trained 200 rounds for a [2, 2] network and 10 training pairs in 13.06 seconds\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3xc5ZXw8d+ZGfXmIrnJRTbuxgUjOgSyJMEmlIQQMM0kkCUQ2BTysiFvlmw+u5vCkiWbEIhfQgg4kJhAnOAkJgRM6LhhG/ciNzXbkq1mdc3Mef+4Y1seS/JImjsjac7385nPbc/ce3wl36N7n/s8j6gqxhhjEpcn3gEYY4yJL0sExhiT4CwRGGNMgrNEYIwxCc4SgTHGJDhfvAPortzcXC0oKIh3GMYY0698+OGHh1U1r6Nt/S4RFBQUsHbt2niHYYwx/YqI7O9smz0aMsaYBGeJwBhjEpwlAmOMSXCWCIwxJsFZIjDGmATnWiIQkadFpEJENneyXUTkZyJSJCIbRWSuW7EYY4zpnJt3BM8A87rYPh+YFPrcBfzCxViMMcZ0wrV2BKr6togUdFHkWmCxOv1grxSRQSIyUlUPuBWTcU8wqLQGgviDSiCoBINKQE9MnXWE5oMEgjjrQtuOlz32fQVFUQUFVI/Nh6ah9cHQek4qG1of+h7t17Xf30n75fj+g6GF8A7a2/fYHt59e5dlT9nWedfv4ZsU5x8rBEGDzrwGEZwpBEFxtged9eixaQAhiBz//rFtJ9Yd3xYMHj+GtItRnDOOnPyPB9Q5RrvlY+tUCdt2bDn0ndCmk9Z1UPb4vHa07kQ8x7dpu32Fzl77fZ+8jZOXT9nfiX10RFTbbWv33ZOKn36fclLxzsqcmB808TzmXHxlhzH1RjwblOUDJe2WS0PrTkkEInIXzl0DY8eOjUlwA1GrP0hNYyvVjW1UNbRS3dhKfbOf+hY/ja1+GloDNLY404YWP42tAVr8AdoCSqs/6HwCYVN/kLZQAujblGT8pNNMOi2kSQsZNJNCG8nSRjJ+kmkjBf9Jy87nxLoUTmzzSQAfAbwEw6YBfBJ0poRPQ+UkgIfgqesJICgeFA9BvNLXz2vfFwQCQJsIfoEAztR/bCqCHyFwyjoIhKb+9lMRAqH5IBAQCKXR4/PO1DlWUJzjayfrg6H9nfS9Y/tuv11g0vYtAy4RSAfrOvytV9UngScBCgsL7X9GB1SViqMtFFXUs/dwAwdqmzhQ00x5bRMHa5s5XN9KfYu/y334PEJGio+MZC8ZKT7Sk72k+LykJnnITvWR7POQ7POS7PU4814JrfOQ7PWS5BN8HsHr8eAV8HoEj0fwyolp+3VeD3jarfO1K+sRQcT5JRENkNRai6+1Fl9rXWhai6/1KN6WWryttXhb6/C21uFpq8fT1oTH34CnrRGPvwlpc+ZFAz0/vwj4UlBvMnhDU08SeH3g8aHiBY/PWedJDc37UI8PxOuUEx/q8YLHe3z78Xk5Nu98VDwE8BAQDxz/iDP1hJaRdtucj4SXDX302HZP+23ek/Ytx76HOMvH/oseKx9aDzjbxJkDEI+02y6ICEFVWtRPc6CN5mArTcE2moPOfHOgjaZgC83BVloCbbSqn9ag82lTPy3BthPLQT+t2n752HxbB8tttKkffzCAX/0EgkGCBHv8c48Gr3jwiBcPgkc8eMUbmjrrnWlo2XNi+eRtznfGTrjKlRjjmQhKgTHtlkcD5XGKpV8JBJWiinrWF1ezoaSGnYeOsquinqPNJy70Xo8wPCuFkYPSODM/h9zMFIZkJDM4I5nB6UkMSU9mUHoy2Wk+MpJ9ZKQ4F/qYCgbg6AGoLYWaEme+oQLqK09M6w9B42HnsUhnfKmQOghScyAlE9IyIGkoJKdDUjokZ7abz3A+x+Z9qeBLAW8K+JLDpingTQZfCuLxhS6OA5Oq0uRvoqGt4aRPfVv9KeuOrW9sa6ShrYHmQDPN/maa/E3H55v9zTQHmnscj098JHmTSPYmk+JJOT6f7Ekm2ZtMki+FNG8WOaF1Sd6kE9s8Sfg8PnweH17xHl/2erz4xHd827Htx5fbbeuorE9C69vNn3xhD03D1vcH8UwEy4D7RGQJcB5Qa/UDnTtQ28RbOyp5a2cl7xYdPn7RH5SexNQRWXxmTj4Th2UycVgm43MzGJ6ditfTBy5b/lao2gNHdsHhnXC4CGr2Oxf+ujII/yvdmwKZwyEzD3JGQ/5ZkDEMModB2uATF/y0QSfmk1Lj82/rg4IapK6ljqrmKmpaaqhrraO2pbbDaV1LHbWttdS11FHXWkcggjsmr3jJSMo4/klPSifNl0ZOcg6pvlTn400lzZfW4XKKN4VUX2jZe2J7+IW+v1xABwrXEoGI/A64DMgVkVLg34EkAFVdBCwHrgSKgEbgi27F0l8dbW7jlU0HeWldKav3VgEwMieVK88cyXkThnDW2MEUDE1HpA9c8FWdC/uBj+DARji4ESp3QPW+ky/2mSNgyHgYe75zoR80BnLGOvPZIyEl+/jjB+NQVapbqjnUcIjKpkqqmqs40nSEquaqk+aPNB+hurm60wu6IGQlZ5GTkkN2cjY5KTmMyhx1fDkzOZPMpEzSk9LJTMo8frFvvy7Fm9I3ft9MVLn51tBNp9muwL1uHb8/O1DbxP97aw9L1hTT3BZkQm4G/+dTk/nUjBFMGpbZN/4j+lugfD3sfx+KV0LZWmg84mwTDwydBCNmwpnXQe5kGDrR+aRmxzfuPkZVqWmpofRoKQcbD3Ko4RCHGkOf0HxFYwVtwbZTvpvmS2NI6hCGpg5lZMZIzsw9kyGpQ45/BqUMOn6Rz07JJis5y/7SNh3qd91QD2QVR5v5yWs7eenDUlTh2jn53Hr+WOaMGRT/i78qHNoCu/4ORSugdA0EWpxtuZNh8nwYNQdGzobhM5zn7waA1kArZfVllB4tpbS+lLKjZZTWlx5fbmhrOKl8sieZ4RnDGZ4+nDnD5jAsfRjD04czIn0EQ9OGOp/UoaQnpcfpX2QGGksEfUAwqLywtoQfLt9Gc1uQBeeM5cuXTmD04Dj/Rw/4Yc8/YPtfYNdrzqMfgBGz4Nx/hrEXOI94MnLjG2cfcbT1KHtq97CnZg97a/c687V7KKsvI9iusjvFm8LozNGMzhpN4YhCRmeOJj8zn1GZoxiePpyclJz4J36TUCwRxFlJVSPffPEjVu+t4vwJQ/jBZ2cyIS8zfgGpQtmHsPH3sPkPzhs7yVlwxsfhsm/DxE84z/ITWFCD7K/bz/aq7cc/u6p3UdlUebxMkieJcdnjmDZkGleOv5Jx2eMYnTWa0ZmjyU3LtQu96VMsEcTRWzsr+dqS9QSCyn9fP4vPnz06fheIpmpY/xys/TVU7Xbe3pkyH2bd6Fz8fcnxiSvOghpkd81uNh3exNYjW9lRtYMd1Tto8jcB4PP4mDRoEheMuoAJOROcz6AJ5Gfm4/PYfy/TP9hvapz8fk0J3/7jJiYNy2TRrWdTkBunZ+pHdsMHP4ePlkBbo/O45+JvwPRrnFczE8zR1qNsqtzEhsoNfFT5ERsrN1LfVg9ARlIGUwZP4bpJ1zFl8BSmDZ3GGTlnkORNinPUxvSOJYI4eOa9vXzvz1u5ZFIuv7j1bDJT4vBjOFwE7/zYeQTkTYKZn4fzvuy86ZNAGtsaWVexjtUHVrPywEq2V21HUTziYeKgicwfP585w+YwK3cWY7PH2ls3ZkCyRBBjv19Twvf+vJVPTR/Oz2+eG/vWvPWV8MZ/wvrfOI9/zr8HLvwqZA2PbRxxEtQgmw5v4t2yd1l9YDUbKzfiVz9JniRm583mntn3MGfYHGbmziQzOY51NcbEkCWCGHqv6DDf/uMmPjY5j8duPiu2ScDfCqsWwduPOI+Azv0yXPJNpwXvANfY1sgH5R/wZumbvF36NlXNVXjEw4yhM7h9xu2cN/I85gybQ5ovLd6hGhMXlghiZP+RBr7y/Dom5mXyxC1zSfF5Y3fwAxvhT1+BQ5tg0qfgU9+HvMmxO34c1LbU8vr+13m9+HVWH1hNa7CVrOQsLs6/mMtGX8ZF+ReRk5J4dSDGdMQSQQy0BYJ8dckGVJVfLiyMXZ1AoA3e+R/nLiB9KCz4LUz9dGyOHQeNbY28UfIGf9v7N94rfw9/0M+YrDHcOPVGPj7m48wZNockj1XsGhPOEkEMPLZiFx+V1PDzm89i7NAYNRKrOwAv3g4lq2DmDTD/YUgfEptjx5Cqsq5iHUt3LeW1/a/R5G9iRMYIbp12K/PHz2fakGn2zr4xp2GJwGU7Dh7l8Td3c91Z+Vw1a1RsDrrvPXjxC9DaAJ/7Fcy8PjbHjaEjTUdYtnsZS3ctZV/dPjKSMvj0hE9z9YSrmTNsjr3dY0w3WCJwkary0MubyUr18W9XTY/NQdf+Gv76TaeHz9uXwbBpsTlujOyo2sFz257jr3v+SluwjbnD5vKlmV/ik+M+aX3vGNNDlghctOyjclbvreIHn53JkAyXW+aqOu0C3vgvp0L4c08NmAZhQQ3yTuk7/Gbrb1h1cBVpvjSum3QdN0+9mQmDJsQ7PGP6PUsELvEHgvzktZ1MHZHFgnPGnP4LvREMwt+/AyufcLqEuPZxp5FYPxfUICuKV7Doo0XsrN7J8PThfH3u17l+8vX2xo8xUWSJwCVL15Wx70gjv1xYiMfNkcJU4a/fgA+fgfPugSt+4Ixp248FNcjr+19n0cZF7KreRUF2AT+4+AfMGz/P3voxxgWWCFzgDwT52Ru7mD06h09MG+buwd74TycJXPR1+MT3+v3oXisPrOTHa37MjuodFGQX8MNLfsj8gvl4PTFsd2FMgnE1EYjIPOCngBd4SlV/FLZ9MPA0cAbQDNyhqpvdjCkW/r71EKXVTTx01XR3X1384AmnncDc2/t9Ethbu5dH1z7Km6VvMipjlCUAY2LIzTGLvcDjwCeBUmCNiCxT1a3tiv1fYIOqflZEpobKX+5WTLHy6/f2MmZIGp+Y5mL/PZtegle/DdOuhqt+0m+TQF1rHU9seIIXtr9Aii+Fr839GrdNv40Ub0q8QzMmYbh5R3AuUKSqewBEZAlwLdA+EUwHfgigqttFpEBEhqvqIRfjctXmslrW7Kvm3z49Da9bdQOHtsDL98HYC+G6p6Cf/tW8Yv8Kvr/q+xxpPsJ1k67j3jn3kptmo50ZE2tuJoJ8oKTdcilwXliZj4DrgHdF5FxgHDAaOCkRiMhdwF0AY8eOdSveqPjt6mLSkrzc4NabQs218MJtzquhn38GklLdOY6LKhsr+cGqH/B68etMGTyFxy5/jBlDZ8Q7LGMSlpuJoKM/hzVs+UfAT0VkA7AJWA/4T/mS6pPAkwCFhYXh++gzmtsC/OWjcuadOYLsVBfeblF1Oo+r3gdf+Eu/6zpaVflT0Z94ZO0jtPhb+Nrcr3H7jNvtTSBj4szNRFAKtP+zeDRQ3r6AqtYBXwQQp1Z1b+jTL72xvYK6Zj/Xzc135wArn3AGkv/U92Hche4cwyW1LbV87/3v8Xrx65w9/Gy+d8H3KMgpiHdYxhjcTQRrgEkiMh4oAxYAN7cvICKDgEZVbQW+BLwdSg790tJ1pQzPTuHCM1x4zn24CFb8B0yeDxfcG/39u2jNwTU8+M6DVDVX8c2zv8nCGQutLyBj+hDXEoGq+kXkPuBVnNdHn1bVLSJyd2j7ImAasFhEAjiVyHe6FY/bahvbeHNHJXdcPD76lcTBIPz5q86IYv3oDSFV5Vebf8Vj6x9jTNYYnrvyOasLMKYPcrUdgaouB5aHrVvUbv4DYJKbMcTKP3ZU4A8q884cEf2df/hr2P8eXPMYZI+M/v5dUN9az7+992+sKF7BFQVX8B8X/od1CmdMH2Uti6Pkta2HyM1MYc7oQdHdcW0pvPbvMP5jcNZt0d23S4rrirl3xb2UHC3hgcIHuG36bTYmgDF9mCWCKGjxB3hrZyVXzRoZ/X6FXvkWaACu/lm/eCS0vmI9X33jqwD88lO/5JwR58Q5ImPM6ViNXRSs3FNFfYufT06P8uuce95y3hK6JDS+QB/3yt5XuPPVO8lJyeH5K5+3JGBMP2GJIAr+sb2C1CQPF02M4ttCwQC8+n8hZyxccF/09uuS57Y+x7++/a/MypvFc/OfY2x23274Z4w5wR4NRcH7uw9zTsEQUpOi2NXDxhfg0Ga4/uk+3XpYVVm0cRFPbHiCT4z9BA9/7GGSvS4PwmOMiSq7I+iliqPN7DxUH922A/5WePOHMHI2zLguevuNMlXlkbWP8MSGJ7j2jGt55NJHLAkY0w/ZHUEvfbD7CAAXnjE0ejtdvxhqiuHTfbfNgKry32v+m+e2Pcct027hX8/5V2skZkw/ZYmgl94vOkJWqo8z86M0dKK/Fd55FMacDxP7Zo/cqsqjHz7Kc9ue47bpt/FA4QP2eqgx/Zj9CddL7+85zPkThkavNfHGJVBXBpc+0GfvBh5b/xjPbHmGBVMWWBIwZgCwRNALB2qbKKlq4oIJUXosFAzAuz+BkXPgjL55N/Dslmf55aZfcv3k6/n2ed+2JGDMAGCJoBfWF9cAMHfc4OjscNufoWoPXHJ/n7wb+Muev/DjtT/mioIreOj8h6xOwJgBwv4n98L64mqSfR6mj8yOzg5XPgGDC2DqVdHZXxS9X/Y+D737EOeMOIcfXPwDSwLGDCD2v7kX1hfXMDM/h2RfFE5j6YdQsgrOu6fPDT25o2oH33jzG0wYNIGffvyn9oqoMQOMJYIeavUH2VRWy1ljotTJ3KpfQEo2nHVLdPYXJUeajvAvb/wLmcmZPHH5E2QlZ8U7JGNMlNnroz20/WAdLf4gZ42NQv1AfSVs+ROccyek9J0LbVugjfvfvJ+q5iqenfcswzP619CYxpjIWCLooWMVxWeNjcIdwYbnIdgGhXf0fl9Roqr816r/Yl3FOh752CPMyLUBZYwZqFx9NCQi80Rkh4gUiciDHWzPEZE/i8hHIrJFRL7oZjzRtLG0lrysFEYNSuvdjoJBZ+CZcRdB3pToBBcFL+58kaW7lnLXrLuYN35evMMxxrjItUQgIl7gcWA+MB24SUSmhxW7F9iqqrOBy4D/EZF+URO5pbyWGaOi8LbQvreheh+c3Xdy4NYjW/nR6h9xUf5F3Dunf42PbIzpPjfvCM4FilR1T2hw+iXAtWFlFMgSp1VSJlAF+F2MKSpa/AGKKuqj89roRy84lcTTru79vqKgrrWO+9+8nyGpQ/jhxT+010SNSQBu/i/PB0raLZeG1rX3c5wB7MuBTcDXVDUYviMRuUtE1orI2srKSrfijdiuQ/X4g8r03t4RtDbCtmUw/do+0dW0qvLQuw9xqOEQP770xwxOjVJDOWNMn+ZmIuioaayGLV8BbABGAXOAn4vIKVdXVX1SVQtVtTAvLy/6kXbT1gN1AMwY1cuO5nYsh9Z6mHVjFKLqvd9t/x1vlLzB/YX3M2fYnHiHY4yJETcTQSkwpt3yaJy//Nv7IrBUHUXAXmCqizFFxdbyOtKTvYwbkt67HW18AbLznYriONtds5tHP3yUS/Iv4dZpt8Y7HGNMDLmZCNYAk0RkfKgCeAGwLKxMMXA5gIgMB6YAe1yMKSq2Hqhj2sjs3g1UX18JRStg5ufBE9/n8G2BNh5850EykjL4j4v+wzqSMybBuNaOQFX9InIf8CrgBZ5W1S0icndo+yLgP4FnRGQTzqOkb6nqYbdiigZVZVt5HZ85K7y6o5u2LAUN9InHQo9teIztVdv52cd/Rm5aFEdaM8b0C642KFPV5cDysHWL2s2XA59yM4ZoK6tp4miLn6kje9kCeOMLMHwmDA9/oza21h5cyzObn+H6ydfz8bEfj2ssxpj4sHcDu2lXRT0Ak4f3IhEcLoKyD2HWDVGKqmea/E38+/v/Tn5mPg8UPhDXWIwx8WNdTHRT0SEnEUzMy+z5Trb+yZme+bkoRNRzT2x4guKjxTz1qadIT+plxbcxpt+yO4Ju2lVxlNzMFAZn9KIB9Pa/Qv7ZkNPLeoZe2FS5icVbF3P95Os5b+R5cYvDGBN/lgi6qaiinonDMnq+g7pyKF8HU66MXlDd1Bpo5bvvf5e8tDzuP/v+uMVhjOkbLBF0g6qyq6KeScN6UT+wI1R3HsdRyJ7e/DRFNUV894Lv2vgCxhhLBN1RcbSFo81+Jg3vRf3A9r/CkDPi1tNoSV0Jv9z4S64ouIKPjf5YXGIwxvQtlgi6YdexiuJhPUwETTWw922Y+um4DE6vqnx/9ffxeXz2lpAx5jhLBN1QVHEUoOePhopeh6A/bo+FVhSv4L2y97h3zr022pgx5jhLBN2wq6KenLQkcjN7+MbQ9r9AxjAYXRjdwCLQ2NbIj1b/iMmDJ3PztJtjfnxjTN9liaAb9h1pYHxuRs/64vG3wK7XYMp88HijH9xpPLnxSQ41HuKh8x/C57HmI8aYEywRdMPeSicR9OzLbztdTsfhsVDJ0RIWb13M1ROutu6ljTGnsEQQoea2AOW1zRQM7WEi2PUa+NJgfOzf1Hl07aP4PD6+NvdrMT+2Mabvs0QQof1HGgEoyO1hVwy7V0DBRTEfiWzNwTW8Xvw6d555p1UQG2M6ZIkgQnsPNwD07NFQ9X44UgRnXB7lqLoWCAZ4ePXDjMwYye0zbo/psY0x/YclggjtO+IkgoKeJILdK5zpxNgmgpd3v8yO6h3cf/b9pPriPyayMaZvskQQoX2HG8jNTCY7Nan7Xy5aAdmjIXdy9APrRLO/mcc3PM6svFlcUXBFzI5rjOl/LBFEaO/hhp5VFAfanDeGJv5TTFsT/3b7b6lorOAbc79hQ08aY7rkaiIQkXkiskNEikTkwQ62PyAiG0KfzSISEJEhbsbUU/uONPTssVDZh9BSF9P6gdqWWp7a9BSX5F9C4YjYN14zxvQvriUCEfECjwPzgenATSJy0riMqvqIqs5R1TnAt4G3VLXKrZh6qrHVz6G6lp5VFBetAPHAhEujH1gnnt78NPWt9fa6qDEmIm7eEZwLFKnqHlVtBZYA13ZR/ibgdy7G02PFVc6ro2OH9ODV0d0rIL8Q0gZHOaqOHWo4xPPbnufKCVcyZUh8ejg1xvQvbiaCfKCk3XJpaN0pRCQdmAf8oZPtd4nIWhFZW1lZGfVAT6c41IZg3NBuJoLGKihbF9O3hRZtXERAA9w3576YHdMY07+5mQg6qqHUTspeDbzX2WMhVX1SVQtVtTAvLy9qAUaqx3cEe/4BaMzqB/bW7uWPu/7IDZNvYHTW6Jgc0xjT/7mZCEqBMe2WRwPlnZRdQB99LARQUtVIVoqPnLRuvjq6921IyYZRZ7kTWJjH1j9GijeFu2bdFZPjGWMGBjcTwRpgkoiMF5FknIv9svBCIpIDXAq87GIsvVJS3cSYIendfw1z7zsw7kLwut/b57Yj23ht/2ssnLGQoWlDXT+eMWbgcC0RqKofuA94FdgG/F5Vt4jI3SJyd7uinwX+rqoNbsXSW8VVjd1/LFRXDlW7oeASd4IKs+ijRWQlZXHb9NticjxjzMDh6p+qqrocWB62blHY8jPAM27G0RvBoFJS1cg/TR3WvS/ue9eZFlwc/aDCbK/azhslb/CV2V8hOznb9eMZYwYWa1l8GpX1LbT4g4wZnNa9L+59G1JzYMRMdwJr59jdwC3Tb3H9WMaYgee0dwQicl0E+2kO/fU/4Bx7Y2hMdx8N7XsXxl3s+mhkO6p2sKJ4BffMvsfuBowxPRLJo6Ff4lTkdlVT+jHCHgENFCU9eXW0thSq98K57r+9c+xu4Nbpt7p+LGPMwBRJInhFVe/oqoCIPBelePqc4qpGRCC/O4+G9r7jTMe7W1G8o2oHrxe/zt2z77a7AWNMj522jkBVT/unZiRl+qviqkZGZKeS4uvGI5597zhdSgyb4V5gOHcDmUmZ3DptwJ5+Y0wMRFxZLCKfF5Gs0Py/ichSEZnrXmh9Q3lNE/mDullRvO8dGHcReNyriz92N3DLtFvISclx7TjGmIGvO1eqh1T1qIhcDFwBPAv8wp2w+o6ymqbuPRaq3g81xa4PUv/kxifJSMqwdgPGmF7rTiIIhKafBn6hqi8DydEPqe8IBJWDtc2M6s4dQQzaD+yr3cdr+19jwZQFdjdgjOm17iSCMhH5f8ANwHIRSenm9/udw/UttAW0e4mg+H1IHQR501yL69dbfk2yN9neFDLGREV3LuQ34HQXMU9Va4AhwAOuRNVHlFY3ATC6O4mgZDWMOc+1+oGDDQdZtnsZn5n4GXLTcl05hjEmsUTcxYSqNgJL2y0fAA64EVRfUV7jJIKI7wgaq+DwTpi9wLWYFm9djKryxTO/6NoxjDGJ5bR/torIumiU6Y9OJILUyL5QssqZjjnPlXhqmmt4aedLzB8/n/zMDsf4McaYbovkjmCaiGzsYrsAA7LGsqymiexUH1mpEY5DULIKPD4Y5c5btb/d/lua/E3ceeadruzfGJOYIkkEUyMoEzh9kf6nvKapmxXFq2DkbEjuwdjGp9HY1sjz257nsjGXMXHwxKjv3xiTuE6bCFR1P4CI/EZVT3ppvaN1A0lZTXPkjcn8rVC+Dgq77I2jx17c+SJ1rXV8aeaXXNm/MSZxdefVlpP6SxARL3B2dMPpW8qqGyNvTHZwI/ibXakfaA20snjrYs4ZcQ6z82ZHff/GmMQWSWXxt0XkKDBLROpCn6NABX14eMneOtrcRl2zP/JHQ8UrnakLieCVva9Q0VhhdQPGGFdE0uncD1U1C3hEVbNDnyxVHaqq3+7quyIyT0R2iEiRiDzYSZnLRGSDiGwRkbd6+O+IugO1zUA3Xh0tWQWDxkL2yKjGoaos3rqYiYMmcuGoC6O6b2OMge49GvqLiGQAiMitIvKoiIzrrHDo0dHjwHxgOnCTiEwPKzMIeAK4RlVnAJ/v7j/ALWWhxmQR1RGoOolgzPlRj2PlgZXsrN7JwukLEelqSAhjjOmZ7iSCXwCNIjIb+FdgP7C4i/LnAkWqukdVW4ElwLVhZW4GlqpqMYCqVnQjHleV1XQjEdTsh/pDMObcqMexeOtihqYO5dMTPh31fRtjDHQvEfhVVXEu5j9V1fSC8k0AABiPSURBVJ8CWV2UzwdK2i2Xhta1NxkYLCJvisiHIrKwox2JyF0islZE1lZWVnYj5J4rr2nC5xHyslJOX7g41JBsbHTvCHbX7Obdsne5aepNJHsHdP9+xpg4iriLCeCoiHwbuA24JPTop6uWVh09x9AOjn82cDmQBnwgIitVdedJX1J9EngSoLCwMHwfriiraWLkoFS8nggex5SshOQsGDb99GW74Tdbf0OqN5UbptwQ1f0aY0x73bkjuBFoAe5Q1YM4f90/0kX5UmBMu+XRQHkHZf6mqg2qehh4G+gT70eW1zQxKifSiuLVMLowqgPVH2k6wp93/5lrzriGwamDo7ZfY4wJF3EiCF38nwdyROQqoFlVu6ojWANMEpHxIpIMLACWhZV5Gefuwici6cB5wLZu/QtcUh5pY7LmWji0JeqPhV7Y8QKtwVbratoY47ruDFV5A7Aa582eG4BVInJ9Z+VV1Q/ch9N19Tbg96q6RUTuFpG7Q2W2AX8DNob2/ZSqbu7pPyZa/IEgB+uaI2tMVroW0KhWFDf7m3lhxwtcOvpSxueMj9p+jTGmI92pI/gOcM6xN3tEJA94HXipsy+o6nJgedi6RWHLj9D1I6aYO3S0hUAwwgFpSlaBeGD0OVE7/l/2/IWq5ipun3F71PZpjDGd6U4dgSfs9c4j3fx+v9GtcQhKVsHwGZDS1QtUkQtqkN9s/Q3ThkyjcHhhVPZpjDFd6c6F/G8i8qqIfEFEvgD8lbC/9geKiBuTBfzOo6EodivxXtl77Kndw23Tb7MGZMaYmDjtoyERmQgMV9UHROQ64GKcV0M/wKk8HnDKIh2QpmILtNZHtUXxs1ufZVj6MOYVzIvaPo0xpiuR3BH8L3AUQFWXqur9qvoNnLuB/3UzuHgpr2licHoS6cmnyZMlq51plCqKd1TtYNWBVdw89WaSvBEOhmOMMb0USSIoUNVTRihT1bVAQdQj6gPKapoie2OoeCVkjXQ6m4uCxVsXk+ZL4/rJnb6MZYwxURdJIujq+Ug3hu/qPyJuTFay2qkfiMKz/IrGCpbvXc5nJ36WnJQBOfKnMaaPiiQRrBGRfw5fKSJ3Ah9GP6T4UlXKqiMYorKuHGqLo1ZRvGT7EgLBALdOswZkxpjYiqQdwdeBP4rILZy48BcCycBn3QosXuqa/DS0Bhh9ukdDxwaiGdv7RNDY1sgLO17g8rGXMyZ7zOm/YIwxURTJmMWHgAtF5OPAmaHVf1XVN1yNLE7KIm1DULIafGkwYlavj7ls9zLqWutYOKPDzleNMcZVEbcsVtV/AP9wMZY+IeLGZCUrIf9s6OXbPccakM3MncmcvDm92pcxxvTEgGwZ3BsRDUjT2gAHNkblsdCbJW9SfLSYhTNsBDJjTHxYIghTXtNEss/D0IwuBoIpWwcaiEpF8eKtixmVMYpPjP1Er/dljDE9YYkgTFlNE6NyUvF0NSBNSaiiuJcdzW05vIUPD33ILdNuwefpTv9/xhgTPZYIwkTUmKxkNeROgfQhvTrWs1ufJSMpg+smXder/RhjTG9YIghz2sZkwaDT42gv6wcONhzk7/v+zucmfY7M5Mxe7csYY3rDEkE7rf4gFUdbun5j6PBOZ1SyXnY09/w2p7++W6bd0qv9GGNMb7maCERknojsEJEiEXmwg+2XiUitiGwIfb7rZjync7C2GVW6fjR0rH6gFxXFDW0NvLTzJT457pOMyhzV4/0YY0w0uFZDKSJe4HHgkziD1K8RkWWqujWs6DuqepVbcXRHRK+OFq+C9KEw9IweH2fprqXUt9WzcLo1IDPGxJ+bdwTnAkWqukdVW4ElwLUuHq/XImpMVrKqVx3N+YN+nt/2PHOHzWVm3swe7cMYY6LJzUSQD5S0Wy4NrQt3gYh8JCKviMiMjnYkIneJyFoRWVtZWelGrMCJO4KROZ10uFpfCVW7e/VY6I3iNyirL7O7AWNMn+FmIujoT2YNW14HjFPV2cBjwJ862pGqPqmqhapamJeXF+UwTyivaSI3M4XUJG/HBUpDA9GM7XlF8bNbn2VM1hguG3NZj/dhjDHR5GYiKAXad6U5GihvX0BV61S1PjS/HEgSkVwXY+pSWU0T+V0NT1m8ErzJMLJnfQJtqNjAxsqN3DrtVryeTpKNMcbEmJuJYA0wSUTGi0gysABY1r6AiIyQUAc7InJuKJ4jLsbUpdM2JitZ7SSBpNOMZdyJxVsXk5WcxWcmfqaHERpjTPS5lghU1Q/cB7wKbAN+r6pbRORuEbk7VOx6YLOIfAT8DFigquGPj2JCVbtuTOZvgfL1PR6fuPRoKSuKV3DD5BtIT0rvRaTGGBNdrnZwE3rcszxs3aJ28z8Hfu5mDJGqbmyjuS3Y+RtD5Rsg0NLj+oHntz2PBw83Tb2pF1EaY0z0WcvikLLqUBuCzh4Nlaxypj14Y6iutY6lu5Yyf/x8hmcM72mIxhjjCksEIadtTFayCgaPh8xh3d73H3b+gUZ/I7dNv603IRpjjCssEYR02ZhMNdTRXPcfC7UF23h+2/OcO+Jcpg2d1tswjTEm6iwRhJTVNJGW5GVwegdDT1btgYbKHlUU/33f3znUeIjbZ9wehSiNMSb6LBGElNc0MWpQasfDRR6vH+jeHYGq8uyWZynILuDi/IujEKUxxkSfJYKQ8pom8gd38lpnySpIyYG8qd3a59pDa9lWtY2FMxbiETvVxpi+ya5OIV22Ki5eBWPOAU/3TtfiLYsZnDKYqydcHYUIjTHGHZYIgOa2AIfrWztuTNZUA5Xbuv1YaE/tHt4sfZMFUxeQ6utZS2RjjIkFSwTAgdpmoJM2BKVrnGk3K4oXb1lMijeFG6fc2NvwjDHGVZYIONGYrMNXR4tXgnhhdGHE+zvcdJg/7/4z15xxDUPThkYrTGOMcYUlAk60IeiwMVnJKhgxE5IzIt7fku1LaAu2WQMyY0y/YIkAp6JYBEaED0gTaIPStd1qSNbkb2LJjiVcNuYyxueMj3KkxhgTfZYIcBLB8KxUkrxhp+PARvA3dSsRvFz0MrUttXxhxheiG6QxxrjEEgEnGpOdomSlM43wjaFAMMDirYuZlTuLs4adFcUIjTHGPZYI6KIxWfEHMGgcZI+MaD9vlrxJydESbp9xe8ctlI0xpg9K+EQQDCrlNc2n3hGoOg3Jxl4Q8b6e2fIM+Zn5XD728ihHaYwx7kn4RHC4oYXWQPDUN4aq9kBDBYyNbPyBDRUb2FC5gYXTF9p4xMaYfsXVRCAi80Rkh4gUiciDXZQ7R0QCInK9m/F0pLzGaUx2SqviYx3NRXhH8OyWZ8lOzrbxiI0x/Y5riUBEvMDjwHxgOnCTiEzvpNzDOGMbx1xpdSPQQavi4g8gNQdyp5x2H8V1xawoXsGNU2608YiNMf2Om3cE5wJFqrpHVVuBJcC1HZT7F+APQIWLsXSq0yEqi1c5bwtF0NHc4q2L8Xl8Nh6xMaZfcjMR5AMl7ZZLQ+uOE5F84LPAIrogIneJyFoRWVtZWRnVIEurm8hO9ZGd2m5AmoYjcHhHRO0HqpurebnoZa6acBV56XlRjc0YY2LBzUTQ0fuTGrb8v8C3VDXQ1Y5U9UlVLVTVwry86F5sy2qaGB3+6ujx+oHTJ4IXdrxAc6CZhdMXRjUuY4yJFZ+L+y4FxrRbHg2Uh5UpBJaE3rnPBa4UEb+q/snFuE5SWt3IuKFh/QiVrARvMoya2+V3m/3N/G7777gk/xImDp7oYpTGGOMeN+8I1gCTRGS8iCQDC4Bl7Quo6nhVLVDVAuAl4CuxTAKqSll1E6NPqR9YCSPnQFLX4wj8qehPVDVX8cUzv+hilMYY4y7XEoGq+oH7cN4G2gb8XlW3iMjdInK3W8ftjprGNhpaAye3IWhrhvL1p30s5A/6eWbLM8zKm0Xh8Mi7qDbGmL7GzUdDqOpyYHnYug4rhlX1C27G0pGyUPfTJ9URlK+HQOtpE8Ere1+hrL6Mb53zLetOwhjTryV0y+JjbQhOejR0vKO5zlsUBzXI05ufZuKgiVw65lI3QzTGGNcleCI4dkfQLhEUr4TcyZCR2+n33ip5i6KaIu448w48ktCn0BgzACT0Vay0uomMZC85aaE2BMGg8+poF3cDqspTm58iPzOf+ePnxyhSY4xxT0IngmNtCI4/4z+8E5qqu6wfWHtoLRsrN/KFGV/A53G1isUYY2IioRNBaXXTyV1L7H/XmY67sNPvPLXpKYakDrHO5YwxA0ZCJ4Ky6saT6wf2vQdZo2Bwx2MNbzmyhffL3+e26beR6uu6jYExxvQXCZsI6prbqGv2n2hDoAr734OCi6CT10F/telXZCZlcuOUG2MYqTHGuCthE0FZdVgbgiO7of4QjLuow/L7avfx+v7XuXHKjWQlZ8UqTGOMcV3CJoLS8O6nj9UPFFzcYflfb/k1yd5kbp1+ayzCM8aYmEnYRFAW3phs33uQMQyGntp53MGGgyzbvYzPTPwMuWmdty8wxpj+KGETQWl1E6lJHoZmJJ+oHxh3YYf1A09tegoU7jjzjjhEaowx7krYRFBW00T+oDSnDUHNfqgr6/Cx0MGGgyzdtZRrJ17LqMxRcYjUGGPclbCJwGlDEKoo3veeM+2govhXm36FqvLPs/45htEZY0zsJGwicFoVH6sofg/ShkDe1JPKHGo4xB92/YFrJ15LfmZ+B3sxxpj+LyETQWOrn6qG1hNtCPa969QPhA1U//Tmp1FVvjTzS3GI0hhjYiMhE0FZ+15Ha0udOoKw+oGKxgpe2vkS10y8htFZo+MRpjHGxISriUBE5onIDhEpEpEHO9h+rYhsFJENIrJWRDp+iT/KjrchGJQGe99xVoYlgqc3P01AA3Y3YIwZ8FxLBCLiBR4H5gPTgZtEZHpYsRXAbFWdA9wBPOVWPO0VVzltCMYOTYc9b0J6LgybcXx7RWMFL+54kWvOuIYxWWNiEZIxxsSNm3cE5wJFqrpHVVuBJcC17Quoar2qamgxA1BiYP+RRtKSvORlJDuJYMKlJ9UPHLsbsDeFjDGJwM1EkA+UtFsuDa07iYh8VkS2A3/FuSs4hYjcFXp0tLaysrLXgRVXNTJ2SDpyeCfUH4TxJ4abLK8v5/c7fm93A8aYhOFmIuioC89T/uJX1T+q6lTgM8B/drQjVX1SVQtVtTAvL6/XgRVXNZx4LAQw4bLj237x0S8QhK/M+Uqvj2OMMf2Bm4mgFGj/J/VooLyzwqr6NnCGiLjamY+qHr8jYO9bztgDg8cBsLtmN8t2L2PB1AWMyBjhZhjGGNNnuJkI1gCTRGS8iCQDC4Bl7QuIyEQJjRMpInOBZOCIizFRebSF5rYgBYOTnTeGJlx2fNtj6x8jzZdmbwoZYxKKa4PuqqpfRO4DXgW8wNOqukVE7g5tXwR8DlgoIm1AE3Bju8pjV+wPvTE0g93QetSpKAY2VW5iRfEKvjLnKwxOHexmCMYY06e4Ovq6qi4HloetW9Ru/mHgYTdjCFd8xEkEBbWrAYGCj6Gq/O+6/2VI6hAWTl8Yy3CMMSbuEq5l8f6qRkRg0MH3YeQsyBjKBwc+YPXB1dw16y4ykjLiHaIxxsRUwiWCkqpGJmQLntI1MOEyVJWfrvspozJG8fnJn493eMYYE3MJlwj2H2ngkxm7IdgG4y/lb/v+xtYjW7lnzj0ke5PjHZ4xxsRcwiWCvYcbuFTWQVI6zfln85MPf8LUIVO5esLV8Q7NGGPiIqESQXVDK9WNrZzZ8AFMuIzf7HqRAw0HeKDwAbweb7zDM8aYuEioRLDncAOTpZSs5gMcHn8xT216in8a80+cO/LceIdmjDFxk1iJoLKeyz3rAXiseR+twVa+WfjNOEdljDHxlVCJYO/hBi73rmfryBn8cf+r3Dz1ZsZmj413WMYYE1cJlQgOHSxntmcX389OZnDqYL48+8vxDskYY+IuoRJB3qF3WZaZxsbWI3yz8JtkJ2fHOyRjjIm7hEkEgaAyvuldHh0yhLnD5trrosYYE5IwiaD8SB0fDd1LvUf4zvnfIdTpqTHGJLyESQSrNjzPn7NTuSrnXCYPnhzvcIwxps9ImESQkpTMrJZk7rnw3+MdijHG9Cnicvf/UVdYWKhr166NdxjGGNOviMiHqlrY0baEuSMwxhjTMUsExhiT4FxNBCIyT0R2iEiRiDzYwfZbRGRj6PO+iMx2Mx5jjDGnci0RiIgXeByYD0wHbhKR6WHF9gKXquos4D+BJ92KxxhjTMfcvCM4FyhS1T2q2gosAa5tX0BV31fV6tDiSmC0i/EYY4zpgJuJIB8oabdcGlrXmTuBVzraICJ3ichaEVlbWVkZxRCNMca4mQg6arrb4buqIvJxnETwrY62q+qTqlqoqoV5eXlRDNEYY4zPxX2XAmPaLY8GysMLicgs4ClgvqoecTEeY4wxHXCtQZmI+ICdwOVAGbAGuFlVt7QrMxZ4A1ioqu9HuN9KYH8Pw8oFDvfwu27qq3FB343N4uoei6t7BmJc41S1w0cqrt0RqKpfRO4DXgW8wNOqukVE7g5tXwR8FxgKPBHqBM7fWcu3dvvt8bMhEVl7uv3HQ1+NC/pubBZX91hc3ZNocbn5aAhVXQ4sD1u3qN38l4AvuRmDMcaYrlnLYmOMSXCJlgj6aoO1vhoX9N3YLK7usbi6J6Hi6ne9jxpjjImuRLsjMMYYE8YSgTHGJLgBmQgi6PVURORnoe0bRWRuDGIaIyL/EJFtIrJFRL7WQZnLRKRWRDaEPt91O67QcfeJyKbQMU8Z9SdO52tKu/OwQUTqROTrYWVidr5E5GkRqRCRze3WDRGR10RkV2g6uJPvdvn76EJcj4jI9tDP6o8iMqiT73b5c3chru+JSFm7n9eVnXw31ufrhXYx7RORDZ1815Xz1dm1Iaa/X6o6oD44bRZ2AxOAZOAjYHpYmStx+jUS4HxgVQziGgnMDc1n4TS2C4/rMuAvcThn+4DcLrbH/Hx18DM9iNMgJi7nC/gYMBfY3G7dfwMPhuYfBB7uye+jC3F9CvCF5h/uKK5Ifu4uxPU94P9E8LOO6fkK2/4/wHdjeb46uzbE8vdrIN4RnLbX09DyYnWsBAaJyEg3g1LVA6q6LjR/FNhG153w9SUxP19hLgd2q2pPW5T3mqq+DVSFrb4WeDY0/yzwmQ6+GsnvY1TjUtW/q6o/tBiXXn07OV+RiPn5OkacVq03AL+L1vEijKmza0PMfr8GYiKIpNfT7vaMGlUiUgCcBazqYPMFIvKRiLwiIjNiFJICfxeRD0Xkrg62x/V8AQvo/D9nPM7XMcNV9QA4/5mBYR2Uife5u4NOevXl9D93N9wXemT1dCePOuJ5vi4BDqnqrk62u36+wq4NMfv9GoiJIJJeTyPuGTXaRCQT+APwdVWtC9u8Dufxx2zgMeBPsYgJuEhV5+IMInSviHwsbHs8z1cycA3wYgeb43W+uiOe5+47gB94vpMip/u5R9svgDOAOcABnMcw4eJ2voCb6PpuwNXzdZprQ6df62Bdt8/XQEwEkfR6GlHPqNEmIkk4P+jnVXVp+HZVrVPV+tD8ciBJRHLdjktVy0PTCuCPOLeb7cXlfIXMB9ap6qHwDfE6X+0cOvaILDSt6KBMvH7XbgeuAm7R0MPkcBH83KNKVQ+pakBVg8AvOzlevM6XD7gOeKGzMm6er06uDTH7/RqIiWANMElExof+mlwALAsrswxYGHob5nyg9tgtmFtCzx9/BWxT1Uc7KTMiVA4RORfn5+Nq19wikiEiWcfmcSoaN4cVi/n5aqfTv9Licb7CLANuD83fDrzcQZlIfh+jSkTm4YztcY2qNnZSJpKfe7Tjal+v9NlOjhfz8xXyCWC7qpZ2tNHN89XFtSF2v1/RrgHvCx+ct1x24tSmfye07m7g7tC84IynvBvYBBTGIKaLcW7ZNgIbQp8rw+K6D9iCU/O/ErgwBnFNCB3vo9Cx+8T5Ch03HefCntNuXVzOF04yOgC04fwVdidOz7krgF2h6ZBQ2VHA8q5+H12OqwjnufGx37NF4XF19nN3Oa7fhH5/NuJcrEb2hfMVWv/Msd+rdmVjcr66uDbE7PfLupgwxpgENxAfDRljjOkGSwTGGJPgLBEYY0yCs0RgjDEJzhKBMcYkOEsExhiT4CwRGGNMgrNEYEwUhFqe/jXUAd5mEbkx3jEZEylfvAMwZoCYB5Sr6qcBRCQnzvEYEzG7IzAmOjYBnxCRh0XkElWtjXdAxkTKEoExUaCqO4GzcRLCDyVGw4waEw32aMiYKBCRUUCVqj4nIvXAF+IckjERs0RgTHTMBB4RkSBOz5b3xDkeYyJmvY8aY0yCszoCY4xJcJYIjDEmwVkiMMaYBGeJwBhjEpwlAmOMSXCWCIwxJsFZIjDGmAT3/wHCMkrvIlxzkgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "network22 = randomNetwork([2,2], 10)\n",
    "\n",
    "plotlistHalf = qnnTraining(network22[0], network22[1], network22[2], 0.5, 0.1, 200)[0]\n",
    "plotlistOne = qnnTraining(network22[0], network22[1], network22[2], 1, 0.1, 200)[0]\n",
    "plotlistTwo = qnnTraining(network22[0], network22[1], network22[2], 2, 0.1, 200)[0]\n",
    "\n",
    "\n",
    "plt.plot(plotlistHalf[0],plotlistHalf[1])\n",
    "plt.plot(plotlistOne[0], plotlistOne[1])\n",
    "plt.plot(plotlistTwo[0], plotlistTwo[1])\n",
    "plt.xlabel(\"s\")\n",
    "plt.ylabel(\"Cost[s]\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Big networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training a 2-3-3-2 network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In training round 0\n",
      "In training round 250\n",
      "In training round 500\n",
      "In training round 750\n",
      "Trained 1000 rounds for a [2, 3, 3, 2] network and 5 training pairs in 241.57 seconds\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3xcdbnv8c+Tya1J27Rp0wtN2xRaaCu2UEIRvCGKlotWFBEQEXUfxC0e3e6jgp7jPudw9vG63erm0lO1IqCCSpUKFTwiguCtpbTQK4Tekt5y6S2XNsnMPPuPWS3TkLQJnZWZzPq+X6+8Zl1+M/MsUtY367d+ay1zd0REJLoKsl2AiIhkl4JARCTiFAQiIhGnIBARiTgFgYhIxBVmu4CBGjt2rNfU1GS7DBGRIeXZZ59tdveq3tYNuSCoqalh5cqV2S5DRGRIMbNtfa1T15CISMQpCEREIk5BICIScQoCEZGICy0IzGyJmTWa2do+1puZfc/M6szseTObF1YtIiLStzCPCO4GFhxn/SXAjODnRuCuEGsREZE+hBYE7v4UsPc4TRYC93jKX4FRZjYxrHpERKR32byOYBJQnzbfECzblZ1yRGQocXfcIeFOIukk3Uk6qemkk/BgWTLVJhm0SRx9pcd86v1Hp4P5I59DsO5IG/dUDUkHJ3gN2rqT1q6XeTj6+c4r70t9xivtes7X1lTyltN7vSbspGQzCKyXZb0+HMHMbiTVfcSUKVPCrEkkbySTTncySVc8SXfC6U6kprsSyaPTqVdPLeuxLpF0upNOIpEknnTiydQOMp5w4slgWbAukXS6E07i6PKgbTIZtD/SJhksf2Vd+nwymdqRv3rHTdrO/pVlUWIGN731tLwLggZgctp8NbCzt4buvhhYDFBbWxuxX78MdfFEko7uBB2dCdq74nR2JzkcT3C4O5Ga7k4E88F08NoZP/Kavi6YjifSduSpHX3XMTv31A45LGZQVFBArMAoLDAKY0asoODodGGBESswimJBm1jB0WUlRQWUHWnb470FZsQKIBZMp+ZfWV5gRkGBEUt/NVLTwbxZ6v3HfgbHfNaR98YKwOzIdGp5gfHK5wfzZkc+C4zUd6Q+JzX/SptXXguCWl41z/HXH/PKse3Cks0gWAbcbGb3A+cBB9xd3UKSE+KJJK2H4xw41M3Bw90cPBQPXrtpPRynvStOR1eC9s4er13xozv8I8s748nXVENxrICSogJKi2KUFhVQWhg7Ol1eXEhlWQHFhQUUxVI/xYUFFMesl2UFFMWM4sJY8Hpk2SvvLy40imMxigpTO+/iWEGwg7bUDj/YuRce2WEXhLdTksEXWhCY2c+AC4GxZtYA/AtQBODui4DlwKVAHdABfDSsWiTaOrritLR10dLexd72Tlrautjbnvppae9iX3vXq3b27V2JE37usKIY5SUxyooLKSuOUV5SyPCSQsaPKKWsJEZ5ceErr8H6YcGOvKQoFuzYj+zog+WFr7zGtLOVQRJaELj7NSdY78Cnwvp+yX+JpLPn4GF2HTh89HX3gUPsPtgZvB6mqbWTw929/0VeXFjAmPJiRpcVUzGsiJqxZYwsLaJiWBEjhxUxsrQweA3mhxUysrSIEaWFlBUXakcteWPI3X1UoiWRdLa1tLOluZ1tLR1sa2ln294Otrd0UL+vg+7Esf3gJYUFTKwoZUJFKedMGU3ViBIqy0sYU17MmOHFVJYXM6a8hMrhxZQXx0LtdxUZKhQEkjNa2jp5YccBNu1uZdOeVl7c08pLe9qO6WMfXlLI1DFlzJw4gne+bgKTK4dxSsUwJlSUMrGilIphRdq5iwyQgkCyojuR5IUdB3hu+35W1+9ndf0+6vceOrp+/MgSzpgwkuvPH8Pp40dw2rjhTK0so7K8WDt6kQxTEMigSCadDbsP8ue6Fp55uZm/b9lLR3BC9pSKUs6aMorrzpvKnOpRzJo4glFlxVmuWCQ6FAQSms54gj+/3MLv1u3h9xv20NTaCcCpVeW8f141F5w2hnOmjmbcyNIsVyoSbQoCyah4Ismf6pr51aod/GFjI22dccqLY1w4cxwXnTGON04fy4QK7fhFcomCQDLipT2t/PLZBpY+t4Om1k5GlRXx7rkTeefsCZx/2hhKi2LZLlFE+qAgkNcsmXSe2NTIj57ZytN1zRQWGG+bOY73z6vmopnjKC7Uc49EhgIFgQzY4e4Ev1hZz5JntrKluZ0JI0v5woIzuKp2MmOHl2S7PBEZIAWB9Nvh7gQPrKjnzj/WsedgJ2dNHsV/XHM2C86cQFFMf/2LDFUKAjmhRNJ5YEU933v8JXYfPMy5NaP596vO4vzTxmhMv0geUBDIcT39UjP/55H1bNzdyjlTR/Ptq+YqAETyjIJAerW1uZ3bHl7P4xsbmVw5jDs/NI9LzpygABDJQwoCOUZ3Isnipzbz3cdfojhWwC2XzOSGC2o0/FMkjykI5Kjntu/j1qUvsHF3K5e+fgL/892v01W/IhGgIBAOdyf4xqOb+NGftzBhZCnfv76Wi2ePz3ZZIjJIFAQRt3bHAT77wGrqGtu4/vypfP5dZzCitCjbZYnIIFIQRFQi6Sx68mW+8/sXqSwv5t6Pz+fNM6qyXZaIZEGoQWBmC4DvAjHgB+7+tR7rRwNLgNOAw8DH3H1tmDUJ1O/t4HM/X82Krfu4bM5E/vW9Z+q2zyIRFubD62PAHcDFQAOwwsyWufv6tGZfAla7+xVmNjNo//awahJ4bN1u/tsv1oDDv39wLu89a5KGhIpEXJhHBPOBOnffDGBm9wMLgfQgmA18FcDdN5pZjZmNd/c9IdYVSV3xJF9/dCM/fHoLc6oruOPaeUyuLMt2WSKSA8IMgklAfdp8A3BejzZrgPcBT5vZfGAqUA0cEwRmdiNwI8CUKVPCqjdv7dh/iE/9ZBWr6/dzwwU13HrpTEoKdV2AiKSEGQS99Td4j/mvAd81s9XAC8BzQPxVb3JfDCwGqK2t7fkZchx/2LiHz/18DfGEc8e187hszsRslyQiOSbMIGgAJqfNVwM70xu4+0HgowCW6qjeEvzISYonknzrdy+y6MmXmT1xJHd+aB41Y8uzXZaI5KAwg2AFMMPMpgE7gKuBa9MbmNkooMPdu4B/AJ4KwkFOwt72Lj79s1U8U9fCNfOn8C/vnq1bRIhIn0ILAnePm9nNwGOkho8ucfd1ZnZTsH4RMAu4x8wSpE4ifzyseqJi7Y4DfOLeZ2lq6+QbV87hqtrJJ36TiERaqNcRuPtyYHmPZYvSpv8CzAizhij59XM7+OKDz1NZXswvPnE+cyePynZJIjIE6MriPBBPJPm/yzey5JktzJ9WyZ0fmqdHRopIvykIhriWtk4+9dNV/HXzXm64oIYvXzZLj40UkQFREAxha3cc4MZ7VtLS3sW/fWAu7z+nOtslicgQpCAYoh5du5t/emA1o8uK+OVNF/D66opslyQiQ5SCYIhxd+568mW+8egmzpo8isXXn8O4EXp4jIi8dgqCIaQznuDWpS+wdNUO3jP3FL5x5RxdHyAiJ01BMES0tHXyiXufZeW2fXzu4tP59EXTdddQEckIBcEQsKW5neuX/I3Gg53cfu3ZXD7nlGyXJCJ5REGQ41bX7+djd68A4IFPnM9ZukhMRDJMQZDDntjUyD/et4qxI4q552PnMU03jRORECgIctSDzzbwhQefZ+aEEfzoo+dqZJCIhEZBkIN++rftfOlXL/DG6WNYdN05jCgtynZJIpLHFAQ55p6/bOUrD63jopnjuPND8zQ8VERCpyDIIT98egu3Pbyei2eP5/Zrz9bjJEVkUCgIcsS9f93GbQ+v55IzJ/C9a87WjeNEZNBob5MDfrNmJ195aC1vnzlOISAig057nCx78sUmPvfz1Zw7tZI7PjRPISAig057nSzauPsg/3jfs0wfN4Lvf6RWJ4ZFJCtCDQIzW2Bmm8yszsxu6WV9hZn9xszWmNk6M/tomPXkkpa2Tv7hxyspLynkRzecS8UwDREVkewILQjMLAbcAVwCzAauMbPZPZp9Cljv7nOBC4F/M7PisGrKFV3xJJ+8bxVNrZ0svr6WCRW6WExEsifMI4L5QJ27b3b3LuB+YGGPNg6MsNRtNIcDe4F4iDXlhK/9diN/37qXb1w5R/cOEpGsCzMIJgH1afMNwbJ0twOzgJ3AC8Bn3D3Z84PM7EYzW2lmK5uamsKqd1D8fv0eljyzhY+cP5WFZ/X8zyEiMvjCDILebpbvPebfBawGTgHOAm43s5GvepP7YnevdffaqqqqzFc6SHYdOMTnf7mG2RNHcuuls7JdjogIEG4QNACT0+arSf3ln+6jwFJPqQO2ADNDrClr3J1bHnyBzniS2689WyOERCRnhBkEK4AZZjYtOAF8NbCsR5vtwNsBzGw8cAawOcSasmbpqh08+WITX3jXGZxaNTzb5YiIHBXaLSbcPW5mNwOPATFgibuvM7ObgvWLgNuAu83sBVJdSV909+awasqWxtbD/O+H11M7dTTXn1+T7XJERI4R6r2G3H05sLzHskVp0zuBd4ZZQy7410c2cKg7wdfeP4eCAj1nWERyi64sDtmz2/bx0OqdfOItpzJ9nLqERCT3KAhClEw6tz28nnEjSrjpradluxwRkV4pCEK0bM1OVtfv5wsLZlJeojt+i0huUhCEpDuR5Fu/28SZk0byvrN14ZiI5C4FQUiWrmqgYd8h/vniM3SCWERymoIgBN2JJP/xhzrmVFdw4RlD90poEYkGBUEIfvXcDhr2HeKz75hB6n56IiK5S0GQYe7O95/azOyJI3nbGeOyXY6IyAkpCDLsTy8181JjGx9/0zQdDYjIkKAgyLAlz2yhakQJl8+dmO1SRET6RUGQQXWNbfxxUxMffsNUSgp1d1ERGRoUBBn0079tpyhmXHvelGyXIiLSbwqCDOmMJ/jVcw1cPHs8Y4eXZLscEZF+UxBkyOMbGtnX0c1VtZNP3FhEJIcoCDLkgRX1nFJRyptn6AIyERlaFAQZsOvAIZ56qYkrz6kmpttJiMgQoyDIgEee34U7XDGvOtuliIgMmIIgA37z/C7OnDSSaWPLs12KiMiAhRoEZrbAzDaZWZ2Z3dLL+s+b2ergZ62ZJcysMsyaMq1+bwdr6vdz+ZxTsl2KiMhrEloQmFkMuAO4BJgNXGNms9PbuPs33f0sdz8LuBV40t33hlVTGB55YRcAl71eVxKLyNAU5hHBfKDO3Te7exdwP7DwOO2vAX4WYj2hePj5ncydPIrJlWXZLkVE5DUJMwgmAfVp8w3BslcxszJgAfBgH+tvNLOVZrayqakp44W+VvV7O1i74yCXvX5CtksREXnNwgyC3sZReh9t3w0801e3kLsvdvdad6+tqsqdcfpPbGoE4OLZCgIRGbrCDIIGIP0y22pgZx9tr2YIdgv9fkMjp44t12ghERnSwgyCFcAMM5tmZsWkdvbLejYyswrgrcBDIdaSce2dcf76cgsXzdTDZ0RkaCs8UQMze18/Puewuy9PX+DucTO7GXgMiAFL3H2dmd0UrF8UNL0C+J27tw+s9Ox6uq6ZrkSSi2YpCERkaDthEADfJ/XX+vHunfAWYHnPhUE49AyIRT3m7wbu7kcdOeXxDXsYUVrIuTVD6rIHEZFX6U8Q/NbdP3a8BmZ2X4bqGRLcnT9uauItp1dRFNPF2SIytJ1wL+bu12WiTT6pa2yjsbWTN08fm+1SREROWr//nDWzD5jZiGD6v5vZUjObF15puevpumYA3qggEJE8MJB+jf/h7q1m9ibgXcCPgbvCKSu3PVPXwpTKMl1NLCJ5YSBBkAheLwPucveHgOLMl5Tb4okkf93coqMBEckbAwmCHWb2/4CrgOVmVjLA9+eFNQ0HaOuM8yYFgYjkiYHsyK8idU3AAnffD1QCnw+lqhz25+D8wPmnjclyJSIimdGf4aMAuHsHsDRtfhewK4yictlfNrcwa+JIKssj1ysmInnqhEcEZrYqE23yQXciyXPb93PeNF1EJiL5oz9HBLPM7PnjrDegIkP15LT1Ow9yqDtBbc3obJciIpIx/QmCmf1okzhxk6FvxdbUXbJ1WwkRyScnDAJ33wZgZve6+4fT1/W2LJ+t3LqPKZVljB9Zmu1SREQyZiCjhl6XPhM8k/iczJaTu9ydFVv3qltIRPJOf04W32pmrcAcMzsY/LQCjQyxZwicjC3N7bS0d6lbSETyTn9uOvdVdx8BfNPdRwY/I9x9jLvfOgg15oSVW/cBcK6OCEQkzwyka+hhMysHMLPrzOzbZjY1pLpyzqrt+6gYVsSpY4dnuxQRkYwaSBDcBXSY2VzgC8A24J5QqspBaxoOMKe6goKC4z2fR0Rk6BlIEMTd3YGFwHfd/bvAiHDKyi2HuhK8uKeVudWjsl2KiEjGDSQIWs3sVuDDwCPBqKGi473BzBaY2SYzqzOzW/poc6GZrTazdWb25ADqGTTrdx0gkXTmVEfiujkRiZiBBMEHgU7gY+6+G5gEfLOvxkFQ3AFcAswGrjGz2T3ajALuBN7j7q8DPjCw8gfHmvoDAMydrCMCEck//Q6CYOf/E6DCzC4HDrv78c4RzAfq3H2zu3cB95PqVkp3LbDU3bcH39E4oOoHyfMN+xk/skQXkolIXhrIoyqvAv5O6q/2q4C/mdmVx3nLJKA+bb4hWJbudGC0mf3RzJ41s+v7+O4bzWylma1samrqb8kZ83zDAebo/ICI5Kl+34Ya+DJw7pG/2s2sCvg98Ms+2vc2vMZ7+f5zgLcDw4C/mNlf3f3FY97kvhhYDFBbW9vzM0J14FA3m5vbed+8nhkmIpIfBhIEBT26blo4/hFFAzA5bb4a2NlLm2Z3bwfazewpYC7wIjnihYbU+QEdEYhIvhrIyeJHzewxM7vBzG4AHgGWH6f9CmCGmU0zs2LgamBZjzYPAW82s0IzKwPOAzYMoKbQrWnYD6ARQyKSt054RGBm04Hx7v55M3sf8CZS3T5/IXXyuFfuHjezm0k93jIGLHH3dWZ2U7B+kbtvMLNHgeeBJPADd1970luVQWt3HGDqmDJGlemJZCKSn/rTNfQd4EsA7r6U4HGVZlYbrHt3X2909+X0OGpw90U95r/JcYahZtuGXQeZPXFktssQEQlNf7qGatz9VU8oc/eVQE3GK8oh7Z1xtu3tYJaCQETyWH+C4HiD54dlqpBctHF3K+4wc0Ik7qQhIhHVnyBYYWb/pedCM/s48GzmS8odG3cfBNARgYjktf6cI/gs8Csz+xCv7PhrgWLgirAKywUbdh1kRGkh1aPz+sBHRCKuP88s3gNcYGZvA84MFj/i7n8ItbIcsGFXK7MmjMRMt54WkfzV7wvK3P0J4IkQa8kpyaSzcddBrjynOtuliIiEaiAXlEVK/b4O2rsSOj8gInlPQdCHDbt0olhEokFB0IcNu1opMDh9vIaOikh+UxD0YcOug9SMLWdYcSzbpYiIhEpB0IcX97TqQjIRiQQFQS8OdyfYvreD6eMUBCKS/xQEvdjS3E7SYca44dkuRUQkdAqCXrzU2AbAdAWBiESAgqAXdY1tFBhMG1ue7VJEREKnIOhFXWMrU8eUU1qkEUMikv8UBL2oa2zjtCp1C4lINIQaBGa2wMw2mVmdmd3Sy/oLzeyAma0Ofr4SZj390Z1IsqW5nRnjFQQiEg39vuncQJlZDLgDuBhoIPVcg2Xuvr5H0z+5++Vh1TFQ21o66E4403VEICIREeYRwXygzt03u3sXcD+wMMTvy4i6YMSQjghEJCrCDIJJQH3afEOwrKfzzWyNmf3WzF4XYj39UtfYCqBzBCISGaF1DQG9Pc3Fe8yvAqa6e5uZXQr8Gpjxqg8yuxG4EWDKlCmZrvMYdY1tTBo1jPKSMP/TiIjkjjCPCBqAyWnz1cDO9AbuftDd24Lp5UCRmY3t+UHuvtjda929tqqqKsSSUxeT6UIyEYmSMINgBTDDzKaZWTFwNbAsvYGZTbDgOZBmNj+opyXEmo7L3dnc1K5uIRGJlND6P9w9bmY3A48BMWCJu68zs5uC9YuAK4FPmlkcOARc7e49u48GzZ6DnRzqTjCtSlcUi0h0hNoRHnT3LO+xbFHa9O3A7WHWMBBbmtsBmDZGQSAi0aEri9NsbUkFQc3YsixXIiIyeBQEabY0t1NcWMApFcOyXYqIyKBREKTZ0txOzZgyCgp6G/kqIpKfFARptja3U6PzAyISMQqCQCLpbGvp0DMIRCRyFASBnfsP0ZVIUqMgEJGIURAEjowY0hGBiESNgiCwtVlBICLRpCAIbG5up6w4xrgRJdkuRURkUCkIAlub25k6ppzg1kciIpGhIAhsbengVHULiUgEKQhIPad4+94O3VpCRCJJQQA07DtEIum6mExEIklBgEYMiUi0KQhIu/20gkBEIkhBQCoIRpQWUllenO1SREQGnYKA1FXF08Zq6KiIRJOCgNQRgbqFRCSqQg0CM1tgZpvMrM7MbjlOu3PNLGFmV4ZZT2864wl27D+kEUMiElmhBYGZxYA7gEuA2cA1Zja7j3ZfJ/WQ+0G3vaUDd50oFpHoCvOIYD5Q5+6b3b0LuB9Y2Eu7TwMPAo0h1tKnIyOGdPtpEYmqMINgElCfNt8QLDvKzCYBVwCLjvdBZnajma00s5VNTU0ZLfLo7afVNSQiERVmEPQ2BMd7zH8H+KK7J473Qe6+2N1r3b22qqoqYwUCbGnuYHRZERVlRRn9XBGRoaIwxM9uACanzVcDO3u0qQXuD4ZtjgUuNbO4u/86xLqOsbW5Xd1CIhJpYQbBCmCGmU0DdgBXA9emN3D3aUemzexu4OHBDAFIdQ2df+qYwfxKEZGcEloQuHvczG4mNRooBixx93VmdlOw/rjnBQbD4e4Euw4cZqrOD4hIhIV5RIC7LweW91jWawC4+w1h1tKbbS0dALr9tIhEWqSvLNbN5kREIh4ER4aO6mSxiERZtIOguZ0x5cWMLNXQURGJrkgHwRYNHRURiXYQbG1p183mRCTyIhsEHV1x9hzsZJpGDIlIxEU2CLY2p4aO6hoCEYm6yAbBthYNHRURgQgHwRYNHRURASIcBFub2xk7vIThJaFeXC0ikvMiHAQdOlEsIkKEg2CLho6KiAARDYLWw900tXYyrUpBICISySCoa2wDYHrV8CxXIiKSfZEOgtPGKQhERKIZBE1tFMWMqZU6WSwiEskgeLkxdaK4MBbJzRcROUYk94QvN7UxXd1CIiJAyEFgZgvMbJOZ1ZnZLb2sX2hmz5vZajNbaWZvCrMegM54gm0t7QoCEZFAaJfVmlkMuAO4GGgAVpjZMndfn9bscWCZu7uZzQF+DswMqyZIXUiWdBQEIiKBMI8I5gN17r7Z3buA+4GF6Q3cvc3dPZgtB5yQHR0xpKGjIiJAuEEwCahPm28Ilh3DzK4ws43AI8DHevsgM7sx6Dpa2dTUdFJF1TW2YaYgEBE5IswgsF6Wveovfnf/lbvPBN4L3NbbB7n7YnevdffaqqqqkyqqrqmNSaOGMaw4dlKfIyKSL8IMggZgctp8NbCzr8bu/hRwmpmNDbEmXtrTqvMDIiJpwgyCFcAMM5tmZsXA1cCy9AZmNt3MLJieBxQDLWEV1BVP8nJTG2dMGBHWV4iIDDmhjRpy97iZ3Qw8BsSAJe6+zsxuCtYvAt4PXG9m3cAh4INpJ48zbnNzG90JZ/bEkWF9hYjIkBPqU1ncfTmwvMeyRWnTXwe+HmYN6TbsOgjALAWBiMhRkbqyeMOuVopjBZyqx1OKiBwVsSA4yIzxw3WPIRGRNJHaI27Y1apuIRGRHiITBE2tnTS3dSoIRER6iEwQbNx95ESxho6KiKSLTBAMK4rxjlnjmDVBRwQiIulCHT6aS2prKvlBTWW2yxARyTmROSIQEZHeKQhERCJOQSAiEnEKAhGRiFMQiIhEnIJARCTiFAQiIhGnIBARiTgL8TkwoTCzJmDba3z7WKA5g+UMBdrmaNA2R8PJbPNUd+/1oe9DLghOhpmtdPfabNcxmLTN0aBtjoawtlldQyIiEacgEBGJuKgFweJsF5AF2uZo0DZHQyjbHKlzBCIi8mpROyIQEZEeFAQiIhEXmSAwswVmtsnM6szslmzXEwYzm2xmT5jZBjNbZ2afCZZXmtn/N7OXgtfR2a41k8wsZmbPmdnDwXy+b+8oM/ulmW0MftfnR2Cb/yn4N73WzH5mZqX5ts1mtsTMGs1sbdqyPrfRzG4N9mebzOxdJ/PdkQgCM4sBdwCXALOBa8xsdnarCkUc+Gd3nwW8AfhUsJ23AI+7+wzg8WA+n3wG2JA2n+/b+13gUXefCcwlte15u81mNgn4r0Ctu58JxICryb9tvhtY0GNZr9sY/H99NfC64D13Bvu51yQSQQDMB+rcfbO7dwH3AwuzXFPGufsud18VTLeS2kFMIrWtPw6a/Rh4b3YqzDwzqwYuA36Qtjift3ck8BbghwDu3uXu+8njbQ4UAsPMrBAoA3aSZ9vs7k8Be3ss7msbFwL3u3unu28B6kjt516TqATBJKA+bb4hWJa3zKwGOBv4GzDe3XdBKiyAcdmrLOO+A3wBSKYty+ftPRVoAn4UdIf9wMzKyeNtdvcdwLeA7cAu4IC7/4483uY0fW1jRvdpUQkC62VZ3o6bNbPhwIPAZ939YLbrCYuZXQ40uvuz2a5lEBUC84C73P1soJ2h3yVyXEG/+EJgGnAKUG5m12W3qqzL6D4tKkHQAExOm68mdWiZd8ysiFQI/MTdlwaL95jZxGD9RKAxW/Vl2BuB95jZVlLdfReZ2X3k7/ZC6t9yg7v/LZj/JalgyOdtfgewxd2b3L0bWApcQH5v8xF9bWNG92lRCYIVwAwzm2ZmxaROsizLck0ZZ2ZGqu94g7t/O23VMuAjwfRHgIcGu7YwuPut7l7t7jWkfqd/cPfryNPtBXD33UC9mZ0RLHo7sJ483mZSXUJvMLOy4N/420md/8rnbT6ir21cBlxtZiVmNg2YAfz9NX+Lu0fiB7gUeBF4GfhytusJaRvfROrw8HlgdfBzKTCG1IiDl4LXymzXGsK2Xwg8HEzn9fYCZwErg9/zr6sUk0IAAAEKSURBVIHREdjm/wVsBNYC9wIl+bbNwM9InQPpJvUX/8ePt43Al4P92SbgkpP5bt1iQkQk4qLSNSQiIn1QEIiIRJyCQEQk4hQEIiIRpyAQEYk4BYGISMQpCEREIk5BIJIBZlZuZo+Y2ZrgnvkfzHZNIv1VmO0CRPLEAmCnu18GYGYVWa5HpN90RCCSGS8A7zCzr5vZm939QLYLEukvBYFIBrj7i8A5pALhq2b2lSyXJNJv6hoSyQAzOwXY6+73mVkbcEOWSxLpNwWBSGa8HvimmSVJ3T3yk1muR6TfdPdREZGI0zkCEZGIUxCIiEScgkBEJOIUBCIiEacgEBGJOAWBiEjEKQhERCLuPwHdLJLnZqTBgAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#mathematica\n",
    "    #run 1: 249,6s\n",
    "#python\n",
    "    #run 1: 148,7s\n",
    "    #run 2: 142,8s\n",
    "\n",
    "network2332 = randomNetwork([2,3,3,2], 5)\n",
    "plotlist2332 = qnnTraining(network2332[0], network2332[1], network2332[2], 4, 0.1, 1000, 250)[0]\n",
    "\n",
    "plt.plot(plotlist2332[0], plotlist2332[1])\n",
    "plt.xlabel(\"s\")\n",
    "plt.ylabel(\"Cost[s]\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training a 2-3-4-3-2 network #1\n",
    "\n",
    "(Small set of training data, high number of rounds.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In training round 0\n",
      "In training round 250\n",
      "In training round 500\n",
      "In training round 750\n",
      "Trained 1000 rounds for a [2, 3, 4, 3, 2] network and 5 training pairs in 895.04 seconds\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXxV9Z3/8dcnNxtbWMO+BBBZRNwibmi1akVpS62Oom2te+1Px07n1zo4baeddubXOnb6s79qpSpqWztiW22lFYutKAVXUBEBQWLCEtawB0LIcj+/P+4FryEJCdyTk9zzfj4eeeSc7/neez9fDed9z27ujoiIRFdW2AWIiEi4FAQiIhGnIBARiTgFgYhIxCkIREQiLjvsAlqrT58+XlRUFHYZIiIdyltvvbXN3QsbW9bhgqCoqIjFixeHXYaISIdiZmubWqZdQyIiEacgEBGJOAWBiEjEKQhERCJOQSAiEnGBBYGZPWpmW81sWRPLzcz+n5mVmNlSMzs1qFpERKRpQW4RPA5Mbmb5pcCo5M+twIMB1iIiIk0I7DoCd/+7mRU102Uq8CtP3Af7dTPrYWYD3H1TUDWJSPvg7sQd6uNO3BM/iWmIx536ZFs8TsqyJl5zsI878YPvkVyOQ9zBcTzZ7pBsT7Q5H01D4vXe4DWJmg9/Dcl+qa+Je6KzkxhLcjb52z/WL/VzDi6LJ58McKj9o/9oFBf14rzjG70m7JiEeUHZIGB9ynx5su2wIDCzW0lsNTB06NA2KU6kvXN3qmvj7K+tp6YuTk1dnAN19Ryoi3MgOV9TH+dAbT019QeXxz/Wt6YuzoH6OHX1Tl19nLq4J6bjTl082X7od8Pp5GsO9jn02jj1cae2/uCK+eCK/aOVvB6DcnRu+8TIjAsCa6St0T8Pd38IeAiguLhYf0LSIbk7+2vrqayuY8/+WvZU17Ln0HQdldW1VFbXsb+mnqqaOqpq6pPT9VTV1rO/Qdv+2vq01JWbnUVOlpEdyyInZsSyjOysLLJjRnbqdCwrOW/k52Ql2rMs2S/ro9+H2oysLCNmid9ZZmQZxA5NG7EsDi37qF+ijyXbYlmkTBuWXB6zZJ+sj/c5+DmW8tsMsgzg4LRhcGialGkzMI7uNVkGfOz1H73GDvU7/HNSX2/J15DyuqCFGQTlwJCU+cHAxpBqEWk1d2f3/lo276lm+94atu09wI59NWzfW8P2fTVsT87v2FfDrv217NlfS128+e8xOTGjU06MzrnZdM6N0Sk3RufcGN075TCgIP9jbZ2SffKzs8jLiZEbyyIvJ4vcWBa52VnkZceSv7M+9jvRL9E/J2ZtsqKR9i3MIJgN3GFms4AzgN06PiDtSX3c2bR7P2u2VbF2xz427apm0+5qNu3ez+bdienGvpVnGfTqkkvvLnn06pLL2IEF9OycQ0F+DgWdcuiWn31ouiA/m275ORR0SrTl58RCGKlEXWBBYGZPAucDfcysHPgukAPg7jOAOcBlQAlQBdwQVC0izamuradk617e37SHD7ZUUratijXb97FuRxU1dfFD/WJZRr9uefTvns/YgQV8ckxfBvToRP+CfPp0zaV318TKv3unHLKy9C1bOo4gzxq65gjLHbg9qM8XaUx1bT1Ly3fz9rqdLN+4h/c37aFs2z7qk7ts8rKzKOrdhRF9unDhmL4U9enCsN6dKerdhb7d8siO6RpMyTwd7jbUIq2xe38tr324jTfKdvD22sTK/+B++kE9OjF2QAGXju/P2AEFjOnfjWG9uxDTt3mJGAWBZJR43Hm3fBcvr6pgweoKlqzfRdwhPyeLCYN7cMt5Izh1aE9OGdqDPl3zwi5XpF1QEEiHVx93Fq/ZwfPLNvOXZZvZvKcaM5gwuAe3X3Ac544q5OQhPcjN1m4dkcYoCKTDKtu2j98uXs/Tb5WztfIAedlZfOL4Qv7lxNFcMLovPTrnhl2iSIegIJAOpa4+zpxlm3nitbW8uWYHsSzjgtGFTD15EJ8c05cuefqTFmkt/auRDmHvgTpmvbmOx15Zw4Zd+ynq3Zm7Jo/milMH068gP+zyRDo0BYG0a3sP1PHowjIeXlBKZXUdZwzvxfennsAFo/vqXH2RNFEQSLtUXVvPE6+v5ecvf8iOfTV8alw/br/gOE4a0iPs0kQyjoJA2p2XVm7lu7OXs25HFZOO68M3LhnNyQoAkcAoCKTdKN9Zxff/tIIXVmxhRGEXfnXjxEBuuSsiH6cgkNC5O7MWrecHf16BO9w1eTQ3Txqh8/5F2oiCQEK1tbKa6U+/x7yVWzl7ZG/+68oJDO7ZOeyyRCJFQSCheaVkG3c++Q57D9Tx3c+M48tnFelMIJEQKAikzbk7D87/kB/PXcXIwq7MuvVMRvXrFnZZIpGlIJA2VVVTx9efWsLc5Vv49IQB3HPFBF0NLBIy/QuUNrNt7wFuenwR723YzbenjOWmScP1mESRdkBBIG2itGIv1z+2iK2V1fziS8VcPK5f2CWJSFKg5+eZ2WQzW2VmJWY2vZHlPc3sD2a21MzeNLPxQdYj4Vi+cTdXzniNvQfqePKWMxUCIu1MYEFgZjHgAeBSYBxwjZmNa9DtX4El7j4BuA74aVD1SDiWbdjNtQ+/QX52Fk9/9WxOGdoz7JJEpIEgtwgmAiXuXuruNcAsYGqDPuOAFwHcfSVQZGb6upghlpbv4tqHX6drXjZPfeUshvfpEnZJItKIIINgELA+Zb482ZbqXeDzAGY2ERgGDG74RmZ2q5ktNrPFFRUVAZUr6fTBlkq+NPNNCjrlMOvWMxnSSxeJibRXQQZBY6eDeIP5HwE9zWwJ8I/AO0DdYS9yf8jdi929uLBQ955p7zbs2s91M98kNzuLJ29RCIi0d0GeNVQODEmZHwxsTO3g7nuAGwAscR5hWfJHOqid+2q4buYb7DtQx1NfOUshINIBBLlFsAgYZWbDzSwXmAbMTu1gZj2SywBuBv6eDAfpgGrq4nzlibdYv3M/D3+5mHEDC8IuSURaILAtAnevM7M7gLlADHjU3Zeb2W3J5TOAscCvzKweWAHcFFQ9Erzv/Wk5b5bt4KfTTubMEb3DLkdEWijQC8rcfQ4wp0HbjJTp14BRQdYgbePXr6/lf95Yx22fGMnUkxueEyAi7Zlu+C7HbNGaHfz77OV8ckxfvnnJ6LDLEZFWUhDIMdlVVcOdT77DoJ6duG/aycR0G2mRDkf3GpKj5u5843dL2bb3AE9/9WwK8nPCLklEjoK2COSoPf7qGv72/hamXzqWCYP1cHmRjkpBIEflgy2V/HDOSi4c05cbzykKuxwROQYKAmm1uvo43/jdu3TNz+aeKyfomQIiHZyOEUirPbSglKXlu3ng2lPp0zUv7HJE5Bhpi0BaZfWWSu7762ouO7E/UyYMCLscEUkDBYG0WDzu3PX0Urrkxfj+VD1DSCRTKAikxX731nreWbeLb08Zp11CIhlEQSAtsquqhh89v5LTi3ry+VN1CwmRTKIgkBb5r7mr2FNdx/enjtdZQiIZRkEgR7S0fBdPvrmOL59VxNgBurW0SKZREEiz3J0f/HkFvbvk8fWLdaNYkUykIJBmvbBiC4vW7OTrF4+im+4lJJKRFATSpNr6OPc8v5KRhV24unjIkV8gIh2SgkCa9NSi9ZRu28f0S8eSHdOfikim0r9uadTeA3Xc97cPmFjUi4vG9g27HBEJUKBBYGaTzWyVmZWY2fRGlnc3sz+Z2btmttzMbgiyHmm5RxaUsm1vDf86ZaxOFxXJcIEFgZnFgAeAS4FxwDVmNq5Bt9uBFe5+EnA+8N9mlhtUTdIyu6tqmbmgjEtO6MfJQ/ScAZFMF+QWwUSgxN1L3b0GmAVMbdDHgW6W+MrZFdgB1AVYk7TAzFfKqDxQxz9ddHzYpYhIGwgyCAYB61Pmy5Ntqe4HxgIbgfeAr7l7vOEbmdmtZrbYzBZXVFQEVa+Q2Bp4bGEZl47vr4vHRCIiyCBobMeyN5i/BFgCDAROBu43s8PWPu7+kLsXu3txYWFh+iuVQx5ZWErlgTruvFAXj4lERZBBUA6knnw+mMQ3/1Q3AM94QglQBowJsCZpxq6qGh57ZQ2XnaitAZEoCTIIFgGjzGx48gDwNGB2gz7rgAsBzKwfMBooDbAmacajC8vYq60BkcgJ7FGV7l5nZncAc4EY8Ki7Lzez25LLZwA/AB43s/dI7Er6F3ffFlRN0rSqmjp+9fpaPjWuH2P6a2tAJEoCfWaxu88B5jRom5EyvRH4VJA1SMs8/VY5u6pqueW8EWGXIiJtTFcWC/VxZ+bCMk4a0oPiYT3DLkdE2piCQPjb+1tYs72KW84drquIRSJIQSA8sqCUQT06MfmE/mGXIiIhUBBE3JL1u1i0Zic3ThquO4yKRJT+5UfcwwtK6ZafzdWn63kDIlGlIIiw9TuqeP69TVx7xlC65gV6ApmItGMKggh77JU1ZJlx/dlFYZciIiFSEETU7v21PLVoHZ85aSADuncKuxwRCZGCIKJmvbmOfTX13Hzu8LBLEZGQKQgiqKYuzmOvrOHskb05YWD3sMsRkZApCCLoufc2snlPNbecq9tJiIiCIHLcnYf/XsZxfbvyieP1bAcRURBEzmsfbmfFpj3cPGk4WVm6nYSIKAgi5+EFpfTpmsvnTmn41FARiSoFQYSUbK3kpVUVfOnMIvJzYmGXIyLthIIgQh5ZUEZedhZfPHNo2KWISDuiIIiIisoDPPPOBq44bTC9u+aFXY6ItCOBBoGZTTazVWZWYmbTG1n+TTNbkvxZZmb1ZtYryJqi6tevr6WmLs5Nk3QBmYh8XGBBYGYx4AHgUmAccI2ZjUvt4+73uvvJ7n4ycDcw3913BFVTVFXX1vPE62u5aGw/RhZ2DbscEWlngtwimAiUuHupu9cAs4CpzfS/BngywHoi6+m3y9mxr4ZbdDsJEWlEkEEwCFifMl+ebDuMmXUGJgNPN7H8VjNbbGaLKyoq0l5oJovHnZkLypgwuDsTh2uvm4gcLsggaOxqJW+i72eAV5raLeTuD7l7sbsXFxbqatjWmLdyK6Xb9nHzuSP0PGIRaVSQQVAOpD72ajCwsYm+09BuoUA8nHwe8WXj9TxiEWlckEGwCBhlZsPNLJfEyn52w05m1h34BPBsgLVE0nvlu3mjbAc3nFOk5xGLSJMCez6hu9eZ2R3AXCAGPOruy83stuTyGcmulwMvuPu+oGqJqocXlNItT88jFpHmHTEIzOzzLXifanef07Ax2TanQduMBvOPA4+34DOkFcp3VvHce5u48ZwiuuXnhF2OiLRjLdkieJjEbpvmjjSeR4MVvoRr5sIyDLjhHJ0yKiLNa0kQPO/uNzbXwcyeSFM9kga7q2p5atF6PnvSQAb20POIRaR5RzyC6O5fTEcfaTtPvLGWqpp6bjlPTyATkSNr8akkZvYPZtYtOf1tM3vGzE4NrjQ5GtW19Tz2yhrOO76QsQMKwi5HRDqA1pxT+B13rzSzScAlwC+BB4MpS47WH9/ZwLa9B/iKtgZEpIVaEwT1yd9TgAfd/VkgN/0lydGKx52HF5RywsACzh7ZO+xyRKSDaE0QbDCzXwBXAXPMLK+Vr5eAzVu5lQ8r9nHrebqdhIi0XGtW5FeRuDhssrvvAnoB3wykKmk1d2fG/A8Z1KMTU04cEHY5ItKBtPjKYnevAp5Jmd8EbAqiKGm910t3sHjtTr4/9QTdTkJEWuWIawwzezsdfSRY97+0msJueVxVrNtJiEjrtGSLYKyZLW1muQHd01SPHIW31u7klZLtfHvKWPJzYmGXIyIdTEuCYEwL+tQfuYsE5f55q+nZOYdrzxgadiki0gEdMQjcfS2Amf3a3b+UuqyxNmlbyzbs5qVVFXzzktF0zg3sZrIiksFac1TxhNSZ5MPpT0tvOdJaP5u3moL8bK47a1jYpYhIB9WSg8V3m1klMMHM9iR/KoGt6GEyoVqxcQ9zl2/h+nOG61bTInLUWnLTuR+6ezfgXncvSP50c/fe7n53G9QoTfjvF1ZRkJ/NTZN0q2kROXqt2TX0ZzPrAmBmXzSzn5iZ9keE5O11O3lx5Va+8omRdO+krQEROXqtCYIHgSozOwm4C1gL/CqQquSIfjx3FX265nLDOUVhlyIiHVxrgqDO3R2YCvzU3X8KdGvuBWY22cxWmVmJmU1vos/5ZrbEzJab2fxW1BNZr5Rs49UPt3P7BcfpTCEROWatWYtUmtndwJeAc5NnDTW5TyK5/AHgYqAcWGRms919RUqfHsDPSdy/aJ2Z9T2aQUSJu3Pv3FUM7J6v6wZEJC1as0VwNXAAuNHdNwODgHub6T8RKHH3UnevAWaR2JpIdS3wjLuvA3D3ra2oJ5L+umILS9bv4msXjSIvW1cRi8ixa3EQJFf+vwG6m9mngWp3b+4YwSBgfcp8ebIt1fFATzN72czeMrPrGnsjM7vVzBab2eKKioqWlpxxauvj/Oj5lYwo7MIVpw4OuxwRyRCteVTlVcCbwD+QuCX1G2Z2ZXMvaaTNG8xnk7gobQqJp559x8yOP+xF7g+5e7G7FxcWFra05Izzm9fXUrptH9+eMlZ3GBWRtGnNMYJvAacf3H1jZoXA34DfN9G/HEi9FeZgYGMjfba5+z5gn5n9HTgJ+KAVdUXC7qpa7ntxNZOO68MFo3UoRUTSpzVfK7Ma7MPffoTXLwJGmdlwM8sFpgGzG/R5lsSB52wz6wycAbzfipoi4/6XVrN7fy3fmjJWTx8TkbRqzRbBX8xsLvBkcv5qYE5Tnd29zszuIPFUsxjwqLsvN7PbkstnuPv7ZvYXYCkQBx5x92VHM5BMtnb7Ph5/dQ1XFw9h7ICCsMsRkQxzxCAws+OAfu7+TTP7PDCJxP7/10gcPG6Su8+hQVi4+4wG8/fS/NlHkfd/5rxPTiyLf/7UYYdPRESOWUt2Dd0HVAK4+zPu/s/u/nUSK/j7gixO4OVVW5m7fAu3X3Acfbvlh12OiGSglgRBkbsf9oQyd18MFKW9Ijmkurae785ezojCLtxy7oiwyxGRDNWSYwTNfQ3tlK5C5HC/mF/K2u1VPHHTGeRm63RREQlGS9Yui8zsloaNZnYT8Fb6SxKAddur+PnLJUyZMIBJo/qEXY6IZLCWbBH8E/AHM/sCH634i4Fc4PKgCosyd+d7f1pOdpbxnSnjwi5HRDJcS55ZvAU428wuAMYnm59z93mBVhZhL6zYwryVW/nWZWPp310HiEUkWC2+jsDdXwJeCrAWIXEF8Xf+uIwx/btxvZ41ICJtQDezb2f+47kVbN9Xw6PXn06O7ickIm1Aa5p2ZP4HFfzurXK+ct4Ixg/qHnY5IhIRCoJ2orK6lrufXsrIwi7ceeGosMsRkQjRrqF24p6/rGTTnmp+f9vZ5OfogTMi0na0RdAOvPbhdp54fR03njOc04b1DLscEYkYBUHI9tfUM/2ZpQzr3ZlvfGp02OWISARp11DI/vuFVazdXsWTt5xJp1ztEhKRtqctghC9vW4nM18p4wtnDOWskb3DLkdEIkpBEJLq2nru+v1SBhTkM/3SMWGXIyIRpl1DIfnZvNWUbN3L4zecTrf8nLDLEZEIC3SLwMwmm9kqMysxs+mNLD/fzHab2ZLkz78FWU97sWzDbmbML+XK0wZzvh5ELyIhC2yLwMxiwAPAxUA5idtZz3b3FQ26LnD3TwdVR3tTUxfnG797l15dcnVnURFpF4LcIpgIlLh7qbvXALOAqQF+XocwY/6HrNxcyX9+bjzdO2uXkIiEL8ggGASsT5kvT7Y1dJaZvWtmz5vZCY29kZndamaLzWxxRUVFELW2iQ+2VPKzeav5zEkD+dQJ/cMuR0QECDYIrJE2bzD/NjDM3U8Cfgb8sbE3cveH3L3Y3YsLCwvTXGbbqI87d/1+KV3zsvneZ7RLSETajyCDoBwYkjI/GNiY2sHd97j73uT0HCDHzDLyuYyPv7qGJet38b3PnkDvrnlhlyMickiQQbAIGGVmw80sF5gGzE7tYGb9zcyS0xOT9WwPsKZQrNtexY/nruKTY/ry2ZMGhl2OiMjHBHbWkLvXmdkdwFwgBjzq7svN7Lbk8hnAlcBXzawO2A9Mc/eGu486NHdn+jNLiWUZ/3n5eJK5JyLSbgR6QVlyd8+cBm0zUqbvB+4Psoaw/Xbxel79cDv/efl4BnTvFHY5IiKH0S0mArRlTzX/8dz7nDG8F9ecPjTsckREGqUgCNB3n11OTV2cH10xgaws7RISkfZJQRCQeSu38Jflm/naRaMY3qdL2OWIiDRJQRCA/TX1fHf2co7r25WbJ40IuxwRkWbp7qMB+PnLJazfsZ8nbzmT3GxlrYi0b1pLpdmHFXv5xfxSLj9lkB42IyIdgoIgjdydf3t2GXk5WfzrZWPDLkdEpEUUBGn03HubeKVkO3ddMprCbrqNhIh0DAqCNKmureeHc1YydkAB154xLOxyRERaTEGQJjMXlrFh136+8+mxxHTNgIh0IAqCNNhaWc3PXyrh4nH9OHtkRt48VUQymIIgDX7ywgfU1Md1gFhEOiQFwTFasXEPTy1ez3VnFekKYhHpkBQEx+iHz79Pj0453PnJUWGXIiJyVBQEx+DVkm0sWL2N2y84Tg+iF5EOS0FwlNyde19YxYDu+XzxTJ0uKiIdl4LgKL34/lbeWbeLOy8cRX5OLOxyRESOmoLgKMTjzo9fWEVR785cedrgsMsRETkmgQaBmU02s1VmVmJm05vpd7qZ1ZvZlUHWky5/WrqRlZsr+frFx5MTU5aKSMcW2FrMzGLAA8ClwDjgGjMb10S/e0g85L7dq6uP83//+gFj+nfjMxMGhl2OiMgxC/Lr7ESgxN1L3b0GmAVMbaTfPwJPA1sDrCVtnl2ykTXbq/jni4/X4ydFJCMEGQSDgPUp8+XJtkPMbBBwOTCjuTcys1vNbLGZLa6oqEh7oS1VH3ceeLmEsQMKuHhcv9DqEBFJpyCDoLGvy95g/j7gX9y9vrk3cveH3L3Y3YsLCwvTVmBrPb9sE6UV+7jjguMw09aAiGSGIB9VWQ4MSZkfDGxs0KcYmJVcqfYBLjOzOnf/Y4B1HZV43Ll/XgkjC7sweXz/sMsREUmbIINgETDKzIYDG4BpwLWpHdx9+MFpM3sc+HN7DAGAF1duZeXmSn5y1Um6zbSIZJTAgsDd68zsDhJnA8WAR919uZndllze7HGB9sTduX/eaob06sRnT9KZQiKSWYLcIsDd5wBzGrQ1GgDufn2QtRyLBau38W75bn74+RPJ1nUDIpJhtFZrgYcXlNK3Wx5XnKqriEUk8ygIjmDV5koWrN7Gl88uIjdb/7lEJPNozXYEMxeW0iknxhfOGBp2KSIigVAQNKOi8gB/fGcjV542mB6dc8MuR0QkEAqCZvz69bXUxuPccE5R2KWIiARGQdCE6tp6nnh9LReO6ceIwq5hlyMiEhgFQRP+8M4Gduyr4aZJw4/cWUSkA1MQNMLdmbmwjBMGFnDmiF5hlyMiEigFQSPmf1BByda93HzucN1cTkQynoKgETMXltGvII8pJ+p2EiKS+RQEDazcvEcXkIlIpGhN18DMBWV0yolx7URdQCYi0aAgSLG1sppnl+gCMhGJFgVBiide0wVkIhI9CoKk6tp6nnhjnS4gE5HIURAkHbyA7OZzdQGZiESLgoDE84hnLixj/KACzhiuC8hEJFoCDQIzm2xmq8ysxMymN7J8qpktNbMlZrbYzCYFWU9T5q9OXEB20yRdQCYi0RPYoyrNLAY8AFwMlAOLzGy2u69I6fYiMNvd3cwmAL8FxgRVU1NmLtAFZCISXUFuEUwESty91N1rgFnA1NQO7r7X3T052wVw2tiKjXtYWKILyEQkuoJc8w0C1qfMlyfbPsbMLjezlcBzwI2NvZGZ3ZrcdbS4oqIirUU+srCUzrkxvjBxWFrfV0SkowgyCBrb2X7YN353/4O7jwE+B/ygsTdy94fcvdjdiwsLC9NW4Obd1fzp3Y1cVTyE7p1z0va+IiIdSZBBUA4MSZkfDGxsqrO7/x0YaWZ9AqzpYx5/dQ31cefGc3TKqIhEV5BBsAgYZWbDzSwXmAbMTu1gZsdZ8jQdMzsVyAW2B1jTIfsO1PE/b6xl8vj+DO3duS0+UkSkXQrsrCF3rzOzO4C5QAx41N2Xm9ltyeUzgCuA68ysFtgPXJ1y8DhQv128nj3Vddx87oi2+DgRkXYrsCAAcPc5wJwGbTNSpu8B7gmyhsbU1ceZubCM04b15NShPdv640VE2pVIni85d/kWynfu5xZtDYiIRC8I3J0H55dQ1LszF4/rF3Y5IiKhi1wQzP+ggmUb9vC/zj+OWJZuJyEiEqkgcHd+Nq+Egd3z+dwph13bJiISSZEKgtdLd/DW2p3cdv5I3U5CRCQpUmvDB14qobBbHlcVDzlyZxGRiIhMELy9bicLS7Zx67kjyM+JhV2OiEi7EZkgcIfzji/k2jOGhl2KiEi7EugFZe3JacN68qsbJ4ZdhohIuxOZLQIREWmcgkBEJOIUBCIiEacgEBGJOAWBiEjEKQhERCJOQSAiEnEKAhGRiLM2ejJk2phZBbD2KF/eB9iWxnI6Ao05GjTmaDiWMQ9z98LGFnS4IDgWZrbY3YvDrqMtaczRoDFHQ1Bj1q4hEZGIUxCIiERc1ILgobALCIHGHA0aczQEMuZIHSMQEZHDRW2LQEREGlAQiIhEXGSCwMwmm9kqMysxs+lh1xMEMxtiZi+Z2ftmttzMvpZs72VmfzWz1cnfPcOuNZ3MLGZm75jZn5PzmT7eHmb2ezNbmfx/fVYExvz15N/0MjN70szyM23MZvaomW01s2UpbU2O0czuTq7PVpnZJcfy2ZEIAjOLAQ8AlwLjgGvMbFy4VQWiDvjf7j4WOBO4PTnO6cCL7j4KeDE5n0m+BryfMp/p4/0p8Bd3HwOcRGLsGTtmMxsE3AkUu/t4IAZMI/PG/DgwuUFbo2NM/rueBpyQfM3Pk+u5oxKJIAAmAiXuXuruNcAsYGrINaWdu29y97eT05UkVhCDSIz1l8luvwQ+F06F6Wdmg4EpwCMpzZk83gLgPGAmgLvXuPsuMnjMSXSkR/wAAALKSURBVNlAJzPLBjoDG8mwMbv734EdDZqbGuNUYJa7H3D3MqCExHruqEQlCAYB61Pmy5NtGcvMioBTgDeAfu6+CRJhAfQNr7K0uw+4C4intGXyeEcAFcBjyd1hj5hZFzJ4zO6+AfgxsA7YBOx29xfI4DGnaGqMaV2nRSUIrJG2jD1v1sy6Ak8D/+Tue8KuJyhm9mlgq7u/FXYtbSgbOBV40N1PAfbR8XeJNCu5X3wqMBwYCHQxsy+GW1Xo0rpOi0oQlANDUuYHk9i0zDhmlkMiBH7j7s8km7eY2YDk8gHA1rDqS7NzgM+a2RoSu/s+aWZPkLnjhcTfcrm7v5Gc/z2JYMjkMV8ElLl7hbvXAs8AZ5PZYz6oqTGmdZ0WlSBYBIwys+FmlkviIMvskGtKOzMzEvuO33f3n6Qsmg18OTn9ZeDZtq4tCO5+t7sPdvciEv9P57n7F8nQ8QK4+2ZgvZmNTjZdCKwgg8dMYpfQmWbWOfk3fiGJ41+ZPOaDmhrjbGCameWZ2XBgFPDmUX+Ku0fiB7gM+AD4EPhW2PUENMZJJDYPlwJLkj+XAb1JnHGwOvm7V9i1BjD284E/J6czerzAycDi5P/nPwI9IzDmfwdWAsuAXwN5mTZm4EkSx0BqSXzjv6m5MQLfSq7PVgGXHstn6xYTIiIRF5VdQyIi0gQFgYhIxCkIREQiTkEgIhJxCgIRkYhTEIiIRJyCQEQk4hQEImlgZl3M7Dkzezd5z/yrw65JpKWywy5AJENMBja6+xQAM+secj0iLaYtApH0eA+4yMzuMbNz3X132AWJtJSCQCQN3P0D4DQSgfBDM/u3kEsSaTHtGhJJAzMbCOxw9yfMbC9wfcglibSYgkAkPU4E7jWzOIm7R3415HpEWkx3HxURiTgdIxARiTgFgYhIxCkIREQiTkEgIhJxCgIRkYhTEIiIRJyCQEQk4v4/zH7OaMNh9kUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#mathematica\n",
    "    #run 1: 2324s = 38,7 min\n",
    "    #run 2: 1501s = 25,0 min\n",
    "#python\n",
    "    #run 1: 545s = 9,1 min\n",
    "    #run 2: 553s = 9,2 min\n",
    "\n",
    "network23432 = randomNetwork([2,3,4,3,2], 5)\n",
    "plotlist23432= qnnTraining(network23432[0], network23432[1], network23432[2], 4, 0.1, 1000, 250)[0]\n",
    "\n",
    "plt.plot(plotlist23432[0], plotlist23432[1])\n",
    "plt.xlabel(\"s\")\n",
    "plt.ylabel(\"Cost[s]\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training a 2-3-4-3-2 network #2\n",
    "\n",
    "(Big set of training data, small number of rounds.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In training round 0\n",
      "In training round 50\n",
      "In training round 100\n",
      "In training round 150\n",
      "In training round 200\n",
      "In training round 250\n",
      "Trained 300 rounds for a [2, 3, 4, 3, 2] network and 100 training pairs in 4833.49 seconds\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXgdddn/8ffdtEnaZmubpGtC0zYtXSgF0hZQFpGlLD4IIpsKilrgEpXHBcEF9edP2VzgUQQReEABKyJLlVJAZBEFbEv3PV2Tpk3TJU3S7Mn9/HFOMaRpm5RMJ+ecz+u6cuXMzPck93TS+Zz5zvI1d0dERBJXr7ALEBGRcCkIREQSnIJARCTBKQhERBKcgkBEJMH1DruArsrOzvaRI0eGXYaISExZsGDBDnfP6WhZzAXByJEjmT9/fthliIjEFDPbdKBl6hoSEUlwCgIRkQSnIBARSXAKAhGRBKcgEBFJcAoCEZEEpyAQEUlwCgIRkR6uvqmF+19fx4JNuwP5+TF3Q5mISKJwd/66ZCt3zF1F6e46rj99NCccNaDbf4+CQESkB3p3825+9NcVLNxcyfihGTz+hcl8aEx2IL9LQSAi0oPsqWvi9hdW8Yd/byY3PYU7L5nMJ44fQVIvC+x3KghERHqIucu2cutzy9lR08AXTyngxjPH0j8l+N20gkBEJGTV9U18/7nlPL1wCxOHZfDQ1VM5ZkTmEfv9CgIRkRAt2LSLG/+4iC276/jqRwv58hlj6J10ZC/oVBCIiITA3fntP9Zzx9zVDMtK5U/XncQJRw0MpRYFgYjIEba3oZmbnlrC80u3cu6kIdx5yWTSU/uEVo+CQETkCCrZVcvnH51H8fYabj73aK49dRRmwV0R1BmBdkSZ2QwzW21mxWZ2cwfLB5jZM2a2xMz+bWaTgqxHRCRMi0squejX/6S8qoHff3461502OvQQgACDwMySgHuBc4EJwBVmNqFds28Di9x9MnAVcE9Q9YiIhOnlFeVc9sBb9E1O4s/XnxzYzWGHI8gjgmlAsbuvd/dGYBZwYbs2E4BXANx9FTDSzAYHWJOIyBH35PwSrv39fMYNTufp6z/EmNy0sEt6nyCDYDhQ0ma6NDqvrcXAxQBmNg04ChjR/geZ2Uwzm29m8ysqKgIqV0Sk+/3urY3c9NQSPjQmmz/MPJGc9JSwS9pPkEHQUceXt5u+HRhgZouALwMLgeb93uT+gLsXuXtRTk5O91cqIhKA37y+jlufW85ZEwbz4NVF9EvumdfnBFlVKZDXZnoEUNa2gbtXAZ8DsMgZkw3RLxGRmPbAG+u47YVVnD95KHdfNoU+R/gmsa4IsrJ5QKGZFZhZMnA5MLttAzPLii4D+ALwRjQcRERi1mNvb+IncyIhcE8PDwEI8IjA3ZvN7AbgRSAJeNjdl5vZddHl9wPjgd+ZWQuwAvh8UPWIiBwJzyws5XvPLeOjR+fyi0unHPHHRRyOQDus3H0OMKfdvPvbvH4LKAyyBhGRI+WVleV8409LOGnUIO791PEk9+75IQAaqlJEpFssKa3khicWMnFYBr+9qojUPklhl9RpCgIRkQ+oZFct1zwyn4H9k3nw6qIjMoZAd4qtakVEepg9tU187pF5NDa3MGvmdHLTU8MuqcsUBCIih6ml1fnKrIVs2rmX339+OmNy08Mu6bAoCEREDtNdL67m9TUV3HbxMZw4alDY5Rw2nSMQETkMsxeXcf/r6/jU9HyumJYfdjkfiIJARKSLVpRVcdNTi5k6cgDf/9jEsMv5wBQEIiJdUNPQzA1PvEtGah9+/akTYuZegYPROQIRkU5yd777zFI27tzLE1/smU8SPRyxH2UiIkfInxaU8uyiMm48c2xMnxxuT0EgItIJa8urufW5ZZw8ehBf+siYsMvpVgoCEZFDqGts4UtPvEtaSm/uvnwKSb3CH2e4O+kcgYjIIfx4zgrWbq/hd9dMi8k7hw9FRwQiIgfx+poKHnt7M1/4cAGnFMbnCIkKAhGRA9hT28RNTy2mMDeNr589LuxyAqOuIRGRA7h19jJ21jTy0NVTY+qx0l2lIwIRkQ78dUkZzy0q46sfLWTS8MywywmUgkBEpJ3tVfV899llHJuXxfWnjw67nMApCERE2nB3vvPsMuoaW/j5pcfGxJjDH1Sga2hmM8xstZkVm9nNHSzPNLO/mNliM1tuZp8Lsh4RkUOZs3QbL68o5+tnj2V0TlrY5RwRgQWBmSUB9wLnAhOAK8xsQrtmXwJWuPuxwOnAz8wsOaiaREQOprK2ke/PXsYxwzO55kMFYZdzxAR5RDANKHb39e7eCMwCLmzXxoF0MzMgDdgFNAdYk4jIAf34+ZXsrm3i9k8ckxBdQvsEuabDgZI206XReW39ChgPlAFLga+6e2uANYmIdOjNtTv404JSrj11FBOHxfdVQu0FGQQdPYzD202fAywChgFTgF+ZWcZ+P8hsppnNN7P5FRUV3V+piCS02sZmbnlmCaOy+/OVjxaGXc4RF2QQlAJ5baZHEPnk39bngKc9ohjYABzd/ge5+wPuXuTuRTk58XmLt4iE5+6/raVkVx23XXxMXN84diBBBsE8oNDMCqIngC8HZrdrsxn4KICZDQbGAesDrElE5H1Wb6vmoTc3cPnUPKbH0RgDXRHYIybcvdnMbgBeBJKAh919uZldF11+P/Aj4BEzW0qkK+lb7r4jqJpERNpqbXW+++xSMlJ7860Z+3VGJIxAnzXk7nOAOe3m3d/mdRlwdpA1iIgcyJ/fLWXext3c+YnJDOifuFeuJ871USIibVTWNnLbC6soOmoAl5wwIuxyQqUgEJGEdMfc1eypa+JHH59ErzgbcayrFAQiknDe3bybWfM287mTRzJ+6H5XrCccBYGIJJTmlla+9+wyBqencuNZY8Mup0dQEIhIQnns7U0sL6vi1o9NIC1FY3OBgkBEEsjOmgZ+9vIaTinM5txJQ8Iup8dQEIhIwvjpS2uoa2zh+x+bQORZlwIKAhFJEMu27GHWvM1cffJIxuSmh11Oj6IgEJG45+788C/LGdgvOSEfKncoCgIRiXuzF5cxb+NuvnnOODL79gm7nB5HQSAica22sZnb5qxi0vAMPlmUd+g3JCBdOyUice2+19axraqeX115HEkJfgfxgeiIQETiVsmuWn7zxnounDKMopEDwy6nx1IQiEjc+vHzK0ky4+ZzE/cR052hIBCRuPTP4h3MXb6NG84Yw9DMvmGX06MpCEQk7rS0Oj/66wryBvbl8x8uCLucHk9BICJx588LSlm1rZqbZ4xPyDGIu0pBICJxpbaxmZ++tJrj8rM47xg9T6gzFAQiEld++8YGtlc38N3zx+t5Qp2kIBCRuLG9up7fvLGOcycN4YSjdLloZwUaBGY2w8xWm1mxmd3cwfJvmtmi6NcyM2sxM209ETksv3h5LY3NrXxrhi4X7YrAgsDMkoB7gXOBCcAVZjahbRt3v8vdp7j7FOAW4HV33xVUTSISv9aUV/PHeZv5zElHMTK7f9jlxJQgjwimAcXuvt7dG4FZwIUHaX8F8IcA6xGROHbbnJX0T+nNV87Q00W7KsggGA6UtJkujc7bj5n1A2YAfz7A8plmNt/M5ldUVHR7oSIS2/5ZvINXV1dww0fGMKB/ctjlxJwgg6Cj0/V+gLYfA/55oG4hd3/A3YvcvSgnJ6fbChSR2Nfa6vz4+ZUMz+rL1SePDLucmBRkEJQCbZ/5OgIoO0Dby1G3kIgchmcWbmHF1ipumjFON48dpiCDYB5QaGYFZpZMZGc/u30jM8sETgOeC7AWEYlD9U0t/PSl1UwekcnHJg8Lu5yYFdh4BO7ebGY3AC8CScDD7r7czK6LLr8/2vQi4CV33xtULSISnx56cwNb99Tzi8um0EtjDRy2QAemcfc5wJx28+5vN/0I8EiQdYhI/NlR08B9r63jrAmDOXHUoLDLiWm6s1hEYtI9f1tLXVOLxhroBgoCEYk5xdtreOLfm7lyWj6jc9LCLifmKQhEJObcMXcVffsk8dUzdfNYd1AQiEhMeXv9Tl5eUc71p48mOy0l7HLigoJARGJGa6vzkzkrGZqZqpHHupGCQERixl+WlLGkdA/fOFs3j3UnBYGIxIT6phbunLuaCUMzuOi4Dh9bJodJQSAiMeHRf21kS2Ud3z1/vG4e62YKAhHp8XbtbeRXrxbzkXE5nDwmO+xy4s4h7yw2s4s78XPqo3cRi4h0u/95ZS17G5q55bzxYZcSlzrziInfEnkg3MGOxU6l3aMkRES6w4Yde3ns7U1cNjWfsYPTwy4nLnUmCF5w92sO1sDMHuumekRE3ufOuatI7t2L/z5LN48F5ZDnCNz9093RRkSkq+Zv3MULy7Zx7amjyU1PDbucuNXpk8Vm9kkzS4++/q6ZPW1mxwdXmogkMnfnx3NWkpuewhdP1c1jQerKVUPfc/dqM/swcA7wKHBfMGWJSKKbs3QbCzdX8o2zx9EvOdAn5ie8rgRBS/T7+cB97v4coFGiRaTbNTS3cMfcVRw9JJ1PnDAi7HLiXleCYIuZ/Qa4FJhjZildfL+ISKf8/q1NbN5Vyy3njSdJN48Fris78kuJDDs5w90rgYHANwOpSkQS1p7aJn7592JOKczmtLE5YZeTEDrd8ebutcDTbaa3AluDKEpEEtevXl1LVX0T39bNY0fMIY8IzOzd7mgjInIom3fW8ui/NnHJ8SMYPzQj7HISRmeOCMab2ZKDLDcgs8MFZjOAe4Ak4EF3v72DNqcDdwN9gB3uflonahKROHTni6vo1Qu+fva4sEtJKJ0Jgs6MDN3SfoaZJQH3AmcBpcA8M5vt7ivatMkCfk3kvMNmM8vtXNkiEm8Wbt7NX5ds5StnjGFIpm4eO5IOGQTuvgnAzH7v7p9pu6yjeW1MA4rdfX207SzgQmBFmzZXAk+7++bo79re9VUQkVjnHhl5LDsthZmnjQ67nITTlauGJradiH7iP+Eg7YcDJW2mS6Pz2hoLDDCz18xsgZld1dEPMrOZZjbfzOZXVFR0oWQRiQUvLi9n3sbd/PdZhaSl6OaxI60zJ4tvMbNqYLKZVUW/qoHtRJ5KesC3djDP2033JhIm5xO5W/l7ZjZ2vze5P+DuRe5elJOjy8lE4kljcyt3zF3FmNw0LivKC7uchNSZh87d5u7pwF3unhH9Snf3Qe5+y0HeWgq03aojgLIO2sx1973uvgN4Azi2i+sgIjHs8Xc2sWHHXr593tH0TtI9qmHoyr/6X82sP4CZfdrMfm5mRx2k/Tyg0MwKzCwZuByY3a7Nc8ApZtbbzPoB04GVXahJRGLYntom7nllLR8aM4iPjNO1ImHpShDcB9Sa2bHATcAm4HcHauzuzcANRO5GXgk86e7Lzew6M7su2mYlMBdYAvybyCWmyw5rTUQk5vzy72vZU9fEd86bgJkeJRGWrpyVaXZ3N7MLgXvc/SEzu/pgb4gOXzmn3bz7203fBdzVhTpEJA5s2rmXR9/ayCdPGMGEYbp5LExdCYJqM7sF+AyR7pwkIjeBiYh02e0vrKJPUi/dPNYDdKVr6DKgAbjG3bcRuRRUn+RFpMvmtRl5bHCGbh4LW6eDILrzfxzINLMLgHp3P+A5AhGRjrS2Ov//+ZUMyUjVyGM9RFeGqryUyAndTxJ5JPU7ZnZJUIWJSHz6y5IyFpdU8o1zNPJYT9GVrfAdYOq+x0CYWQ7wN+CpIAoTkfhT39TCnXNXM3FYBhcf1/5BAxKWrpwj6NXuWUA7u/h+EUlwD725gS2VdXzn/PH00shjPUZXjgjmmtmLwB+i05fR7tJQEZEDqahu4L7X1nHm+MGcPDo77HKkjUMGgZmNAQa7+zfN7GLgw0SeI/QWkZPHIiKHdOfcVTQ0t/Dt8zrzZHs5kjrTtXM3UA3g7k+7+9fc/b+JHA3cHWRxIhIfFm7ezZ8WlHLNhwsYlZMWdjnSTmeCYKS77zdCmbvPB0Z2e0UiEldaW50fzF5ObnoKXz6jMOxypAOdCYKD3e3Rt7sKEZH49NS7pSwu3cMt5x2tsQZ6qM4EwTwz+2L7mWb2eWBB95ckIvGiqr6JO+eu4vj8LD4+RZeL9lSdiecbgWfM7FP8Z8dfBCQDFwVVmIjEvnv+tpadext55HPT9HTRHqwzYxaXAyeb2UeASdHZz7v73wOtTERi2tryah7910Yun5rPpOGZYZcjB9HpDjt3fxV4NcBaRCROuDs//MsK+iUn8Y2z9xt9VnoY3RksIt3uhWXbeLN4B187ayyD0lLCLkcOQUEgIt2qqr6JH8xezqThGXz6xIONZis9ha7lEpFu9dMXV7OjpoEHry7SYPQxQltJRLrNopJKfv/2Jq46aSSTR2SFXY50UqBBYGYzzGy1mRWb2c0dLD/dzPaY2aLo161B1iMiwWluaeWWp5eSm57C13WCOKYE1jUUHdP4XuAsoJTIjWmz3X1Fu6b/cPcLgqpDRI6M//3nRlZureL+Tx9PeqqGM48lQR4RTAOK3X29uzcCs4ALA/x9IhKSkl21/PzlNZw5PpdzJg4JuxzpoiCDYDhQ0ma6NDqvvZPMbLGZvWBmEzv6QWY208zmm9n8ioqKIGoVkcPk7tz89BJ6GfzgvybqDuIYFGQQdPTX4O2m3wWOcvdjgV8Cz3b0g9z9AXcvcveinJycbi5TRD6Ix9/ZzD+Ld/Kd8ycwYkC/sMuRwxBkEJQCeW2mRwBlbRu4e5W710RfzwH6mJmGLhKJESW7avnJnJWcUpjNFdPyDv0G6ZGCDIJ5QKGZFZhZMnA5MLttAzMbYtHjSDObFq1nZ4A1iUg3aW11bnpqCb3MuP0Tk9UlFMMCu2rI3ZvN7AbgRSAJeNjdl5vZddHl9wOXANebWTNQB1zu7u27j0SkB3r8nU28tX4nt198DMOzNDRJLAv0zuJod8+cdvPub/P6V8CvgqxBRLrfpp17ue2FVZw6NofLpqpLKNbpzmIR6ZKmlla+MmsRSb2M2y8+Rl1CcUDPGhKRLrn7b2tYXFLJvVcezzB1CcUFHRGISKf9a90Ofv3aOi4ryuP8yUPDLke6iYJARDpl995GvvbHxRQM6s/3/2tC2OVIN1IQiMghtbY6N/5xEbv2NvI/VxxHv2T1KscTBYGIHNIv/17M62squPVjEzT+cBxSEIjIQb2+poK7X1nDxccN51PT88MuRwKgIBCRAyrdXctXZy1k3OB0fnyRLhWNVwoCEenQ3oZmvvDofFpanF9/6nj6JieFXZIERGd8RGQ/+04Orymv5uHPTmVUTlrYJUmAdEQgIvu566XVvLyinO9dMIHTx+WGXY4ETEEgIu/z5PwS7nttHVdOz+ezJ48Muxw5AhQEIvKeV1aWc8vTSzmlMJsfarSxhKEgEBEAFmzazZeeeJcJQzO479Mn0CdJu4dEoS0tIqwtr+bzj85jSEYq//u5qaSl6DqSRKIgEElw6ypquPLBd+iT1IvfXTOd7LSUsEuSI0xBIJLANuzYyxUPvI2784cvTid/kAafT0Q6/hNJUJt27uXK375Nc6sza+aJjMlND7skCYmCQCQBrd5WzWceeoemllae+OKJjB2sEEhk6hoSSTDvbt7Npb95CzN48tqTGD80I+ySJGSBBoGZzTCz1WZWbGY3H6TdVDNrMbNLgqxHJNG9vqaCTz/4Dln9+vDUdSdTqCMBIcAgMLMk4F7gXGACcIWZ7TesUbTdHcCLQdUiIvD4O5u45pF55A/sx5+uPYm8gToxLBFBniOYBhS7+3oAM5sFXAisaNfuy8CfgakB1iKSsFpandtfWMlv/7GB08fl8MsrjiM9tU/YZUkPEmQQDAdK2kyXAtPbNjCz4cBFwBkcJAjMbCYwEyA/XwNjiHTWnromvvbHRbyyajtXnXQUt14wgd66Y1jaCTIIOnpIibebvhv4lru3HOyZJu7+APAAQFFRUfufISIdWLm1iuseW8CW3XX88L8mctVJR+nZQdKhIIOgFMhrMz0CKGvXpgiYFf3jzAbOM7Nmd382wLpE4pq789SCUr733DIy+/bhj9eeyAlHDQy7LOnBggyCeUChmRUAW4DLgSvbNnD3gn2vzewR4K8KAZHDV1nbyHeeWcbzS7cyvWAgv7zyOHLTU8MuS3q4wILA3ZvN7AYiVwMlAQ+7+3Izuy66/P6gfrdIIvpX8Q6+9uRidtQ0cNOMcVx76miSeqkrSA4t0DuL3X0OMKfdvA4DwN0/G2QtIvFqb0MzP3tpDf/7rw0UZPfnmas+xDEjMsMuS2KIHjEhEsP+vqqc7z27nC2VdXzmxKO45byj6Zes/9bSNfqLEYlB26vq+eFfVvD80q0U5qbx1HUnUTRSJ4Tl8CgIRGJIfVMLD725gV+/WkxTq/ONs8cy89TRJPfWvQFy+BQEIjHA3Zm9uIw7565mS2UdZ08YzC3njacgu3/YpUkcUBCI9GDuzmtrKrj75TUsLt3DxGEZ/PSTx3LS6EFhlyZxREEg0gO5O/9Yu4Nf/G0NCzdXMjyrL3ddMpmLjx+hS0Kl2ykIRHqQllbn5RXlPPDGOt7dXMmwzFR+ctExXHLCCJ0HkMAoCER6gLrGFp5aUMKDb25g085a8gb25Ucfn8SlRSNI6Z0UdnkS5xQEIiEq2VXLrHmbefydzVTWNjElL4tvzTiacyYOUReQHDEKApEjrKmllVdWbueJf2/mH2srMODM8YOZeeooTjhqgJ4QKkecgkDkCFlbXs0zC7fwpwWlVFQ3MCQjla+cUcilU/MYntU37PIkgSkIRAK0bU89sxdv4dmFZazYWkUvg4+My+WKafmcPi5Hg8RIj6AgEOlm2/bU89KKbbywdBtvb9iJOxw7IpNbL5jABccO1WOhpcdREIh0g3UVNby4fBsvLi9ncUklAKNz+vOVMwq5cMowRuWkhVyhyIEpCEQOQ31TC/M27uL11RW8tqaC4u01QOST/zfPGcc5E4cwJlc7f4kNCgKRTnB31u/Yy+urK3hjbQVvr99JfVMryUm9mFowgE9Pz+fsiUMYppO+EoMUBCIdcHdKdtXx9oad/HvDLt5at5MtlXUAjMruz+VT8zltbA7TRw3U8/8l5ukvWITIjn9dxV7eie7431m/i21V9QAM6NeHaQUDuf700Zw2Noe8gf1CrlakeykIJCFV1zexpHQPi0oqWVRSycLNu9lR0whATnoK0wsGRr5GDWJMThq9dJevxLFAg8DMZgD3EBm8/kF3v73d8guBHwGtQDNwo7u/GWRNkniaW1pZXV4d2elvjuz4iytqcI8sL8juz6mFOUwrGMi0goEUZPfX3b2SUAILAjNLAu4FzgJKgXlmNtvdV7Rp9gow293dzCYDTwJHB1WTxL/6phZWb6tmxdYqlpftYUVZFSu3VlPX1AJEunmm5GVxweRhHJuXyZS8LLL6JYdctUi4gjwimAYUu/t6ADObBVwIvBcE7l7Tpn1/wAOsR+LMrr2NrCirYsXWyA5/eVkV6ypqaI3+FaWl9GbC0Awum5rHcflZTMnLIn9gP33aF2knyCAYDpS0mS4FprdvZGYXAbcBucD5AdYjMWpPbRNrtleztryGNeXVFG+PfN9e3fBem6GZqUwYmsG5k4YwYVgGE4ZmMmJAX/Xti3RCkEHQ0f/A/T7xu/szwDNmdiqR8wVn7veDzGYCMwHy8/O7uUzpKSprG1lTXsPa6E5/7fZq1pTXUNFmh98vOYnC3DROKcxh3JA0Jg7LZPzQDAb2V/eOyOEKMghKgbw20yOAsgM1dvc3zGy0mWW7+452yx4AHgAoKipS91EMq29qYePOvWyo2MuG6PeNO/eyYcfe967aAeifnMSYwemcNjaHsYPTKMxNp3BwGsMy9SlfpLsFGQTzgEIzKwC2AJcDV7ZtYGZjgHXRk8XHA8nAzgBrkiOgqaWV0t11bNhRw/o2O/oNFXsp21P/vrY56SkUZPfnzPGDGZXTn8LB6YwdnM6wzFT15YscIYEFgbs3m9kNwItELh992N2Xm9l10eX3A58ArjKzJqAOuMzd9Yk/BuypbWLzrlpKdtdGvu/6z/eS3XW0tP5nM2ak9qYgJ43powZRkN2fkdn9GRX9npaiW1lEwmaxtt8tKiry+fPnh11G3GtobmHL7jpKdtf9ZwffZmdfVd/8vvZZ/fqQP7AfeQP6MTK7HyMH9WdUTn8KstMY0K+PPt2LhMzMFrh7UUfL9HEsQdU3tVBWWUdZZT1llXWUVtZFd/yRHf22qnrafkZI7t2LEQP6kj+wH8fnD4js9Af2JW9gP/IG9iMjtU94KyMiH4iCIA65O7trm9iyu44tlXWUVf7n+77XbU/MApjB4PRU8gb25aTRg8gb0I/8gf3IHxT5lJ+bnqKTtCJxSkEQg+qbWiivqn/v0/y+nfuWNjv8+qbW972nb58khmWlMnxAPyYMy2BYZl+GD+jLsKy+DM/qy5DMVPpo2ESRhKQg6GFqGprZtqeOrXvq2bqnnvI99WytqmfbvumqenbtbdzvfdlpKQzPSuXoIemcMS73fTv54Vl9yVI/vYgcgILgCNnXXbN1Tx3lVZGd+rZ9X22maxqa93vvoP7JDMlMZXhWKsfnZzE0M5UhmX0ZkpHK8AF9GZqZSmqfpBDWSkTigYKgG9Q2NrO9qoHt1Q2UV9WzvbqB7VX/2cHv2/E3Nr+/u6aXweCMVIZkpkbvls1maGYqgzNSGZoZ2cHnZqSQ0ls7eREJjoLgIGobmymviuzUy6M79307+fKqBrZX17O9qoHqDj7FJ/fuxZDoTn5KXhYzJqUyJCP1fZ/ms9OS6a1+eREJWUIGwd6G5v0+vb83XdVAeXU9FQfYwaf07sXgjFRy01M4ekgGpxSmvDc9OCPyCX5weioZfXurT15EYkLCBMGrq7fzo7+sYHt1Q4f98Pt28IMzUhg/JIPTxqaQmx6ZbvtdO3gRiTcJEwRZffswflgGp6W3+wSfnkJuRioZqdrBi0hiSpggOC5/APdeOSDsMkREehydqRQRSXAKAhGRBKcgEBFJcAoCEZEEpyAQEUlwCgIRkQSnIBARSXAKAhGRBBdzYxabWQWw6TDfng3s6MZywqR16ZniZV3iZT1A67LPUf0siVEAAAQhSURBVO6e09GCmAuCD8LM5h9o8OZYo3XpmeJlXeJlPUDr0hnqGhIRSXAKAhGRBJdoQfBA2AV0I61LzxQv6xIv6wFal0NKqHMEIiKyv0Q7IhARkXYUBCIiCS5hgsDMZpjZajMrNrObw67ngzCzjWa21MwWmdn8sOvpCjN72My2m9myNvMGmtnLZrY2+r3HjyB0gPX4gZltiW6XRWZ2Xpg1dpaZ5ZnZq2a20syWm9lXo/NjarscZD1ibruYWaqZ/dvMFkfX5YfR+YFsk4Q4R2BmScAa4CygFJgHXOHuK0It7DCZ2UagyN1j7iYZMzsVqAF+5+6TovPuBHa5++3RkB7g7t8Ks85DOcB6/ACocfefhllbV5nZUGCou79rZunAAuDjwGeJoe1ykPW4lBjbLhYZN7e/u9eYWR/gTeCrwMUEsE0S5YhgGlDs7uvdvRGYBVwYck0Jyd3fAHa1m30h8Gj09aNE/vP2aAdYj5jk7lvd/d3o62pgJTCcGNsuB1mPmOMRNdHJPtEvJ6BtkihBMBwoaTNdSoz+gUQ58JKZLTCzmWEX0w0Gu/tWiPxnBnJDrueDuMHMlkS7jnp0V0pHzGwkcBzwDjG8XdqtB8TgdjGzJDNbBGwHXnb3wLZJogSBdTAvlvvEPuTuxwPnAl+KdlNI+O4DRgNTgK3Az8Itp2vMLA34M3Cju1eFXc/h6mA9YnK7uHuLu08BRgDTzGxSUL8rUYKgFMhrMz0CKAuplg/M3cui37cDzxDp+opl5dH+3X39vNtDruewuHt59D9vK/BbYmi7RPuh/ww87u5PR2fH3HbpaD1iebsAuHsl8Bowg4C2SaIEwTyg0MwKzCwZuByYHXJNh8XM+kdPhGFm/YGzgWUHf1ePNxu4Ovr6auC5EGs5bPv+g0ZdRIxsl+iJyYeAle7+8zaLYmq7HGg9YnG7mFmOmWVFX/cFzgRWEdA2SYirhgCil4zdDSQBD7v7j0Mu6bCY2SgiRwEAvYEnYmldzOwPwOlEHqdbDnwfeBZ4EsgHNgOfdPcefSL2AOtxOpHuBwc2Atfu68/tyczsw8A/gKVAa3T2t4n0r8fMdjnIelxBjG0XM5tM5GRwEpEP7E+6+/8zs0EEsE0SJghERKRjidI1JCIiB6AgEBFJcAoCEZEEpyAQEUlwCgIRkQSnIBARSXAKAhGRBKcgEOkG0Tu+n48+P36ZmV0Wdk0indU77AJE4sQMoMzdzwcws8yQ6xHpNB0RiHSPpcCZZnaHmZ3i7nvCLkiksxQEIt3A3dcAJxAJhNvM7NaQSxLpNHUNiXQDMxtGZAjBx8yshsgwjyIxQUEg0j2OAe4ys1agCbg+5HpEOk1PHxURSXA6RyAikuAUBCIiCU5BICKS4BQEIiIJTkEgIpLgFAQiIglOQSAikuD+D5wTOiiYWVHNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#mathematica\n",
    "    #(430s for 50/50)\n",
    "    #run 1: 4762s = 79 min\n",
    "#python\n",
    "    #(266s for 50/50)\n",
    "    #run 1: 3328s = 55 min\n",
    "\n",
    "network23432Two = randomNetwork([2,3,4,3,2], 100)\n",
    "plotlist23432Two= qnnTraining(network23432Two[0], network23432Two[1], network23432Two[2], 4, 0.1, 300, 50)[0]\n",
    "\n",
    "plt.plot(plotlist23432Two[0], plotlist23432Two[1])\n",
    "plt.xlabel(\"s\")\n",
    "plt.ylabel(\"Cost[s]\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Generalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Subset training for a 2-2 network | D = 4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "n=1, i=0\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 14.72 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 11.71 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 9.94 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 10.21 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 9.79 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 10.17 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 9.72 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 9.22 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 9.56 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 11.55 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 8.87 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 11.31 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 11.58 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 9.47 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 8.98 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 9.85 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 8.9 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 9.59 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 24.4 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 1 training pairs in 17.21 seconds\n",
      "n=2, i=0\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 29.93 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 18.01 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 23.54 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 32.08 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 26.18 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 18.45 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 17.62 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 28.41 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 32.28 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 28.82 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 37.85 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 21.38 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 17.9 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 28.44 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 20.91 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 17.43 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 17.56 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 18.09 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 17.61 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 2 training pairs in 18.78 seconds\n",
      "n=3, i=0\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 27.78 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 39.89 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.45 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 34.38 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 27.54 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.48 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 27.7 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 24.35 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.59 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.41 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.52 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.41 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.94 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 24.54 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.52 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.46 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 23.25 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 24.35 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 24.46 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 3 training pairs in 28.2 seconds\n",
      "n=4, i=0\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 29.66 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 29.98 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 31.52 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 31.55 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 29.08 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 29.32 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 32.25 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 31.18 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 33.0 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 29.27 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 28.33 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 27.98 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 27.62 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 27.95 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 28.31 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 28.59 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 27.78 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 28.76 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 32.02 seconds\n",
      "Trained 1000 rounds for a [2, 2] network and 4 training pairs in 28.77 seconds\n",
      "1811.1571247577667\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAASGklEQVR4nO3df2zc933f8eeLss2MSGdns8JmkiVagOtVLZYs47QUxTpjW1s57WZ0yB92uWXzfggO6iLDAK9uhaXYBgEbhg1NW28C0RlJMC7GgGSpFyjzhm1uhkJtTSdOYsdlIKiTzDqgmQRL1jClIem9P+6cUvSRPIpHHu+j5wMgeJ8fvHt/9JFe/PB7d1SqCknS6BsbdgGSpMEw0CWpEQa6JDXCQJekRhjoktSIW4b1wHfeeWdNTU0N6+ElaSQ9//zzX6uqg73GhhboU1NTzM/PD+vhJWkkJbm00ZiXXCSpEQa6JDXCQJekRhjoktQIA12SGrFloCd5MslrSV7cYDxJfjnJhSRfTPLuwZcpSaNvbmmJqfPnGXv2WabOn2duaWmg99/PCf0jwMlNxu8H7ul+nAL+3c7LkqS2zC0tcWphgUurqxRwaXWVUwsLAw31LQO9qj4LfGOTKQ8AH6uO3wLuSPKOQRUoSS04ffEiK9euXde3cu0apy9eHNhjDOIa+iHglTXtxW7fmyQ5lWQ+yfzy8vIAHlqSRsPl1dVt9d+IQQR6evT1/F8zqmq2qqaravrgwZ7vXJWkJh0ZH99W/40YRKAvAnetaR8GXh3A/UpSM84cO8bE2PWROzE2xpljxwb2GIMI9KeB93df7fIe4JtV9dUB3K8kNWNmcpLZe+/l6Pg4AY6OjzN7773MTE4O7DG2/OVcST4O3AfcmWQR+EXgVoCqOgucA94LXABWgIcHVp0kNWRmcnKgAb7eloFeVQ9tMV7AzwysIknSDfGdopLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiStEfm5mBqCsbGOp/n5gZ7/1u+U1SStHNzc3DqFKysdNqXLnXaADMzg3kMT+iStAdOn/6jMH/Dykqnf1AMdEnaA5cvb6//RhjokrQHjhzZXv+NMNAlaQ+cOQMTE9f3TUx0+gfFQJekPTAzA7OzcPQoJJ3Ps7ODe0IUfJWLJO2ZmZnBBvh6ntAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSI/oK9CQnkywkuZDk8R7jb0vyn5N8McnvJPnBwZcqSdrMloGe5ADwBHA/cBx4KMnxddN+AXihqv4M8H7gw4MuVJK0uX5O6CeAC1V1sapeB54CHlg35zjwPwCq6neBqSSTA61UkrSpfgL9EPDKmvZit2+tLwB/AyDJCeAocHgQBUqS+tNPoKdHX61r/wvgbUleAH4W+Dxw5U13lJxKMp9kfnl5edvFSpI21s/vclkE7lrTPgy8unZCVX0LeBggSYDf636wbt4sMAswPT29/puCJGkH+jmhPwfck+TuJLcBDwJPr52Q5I7uGMDfBz7bDXlJ0h7Z8oReVVeSPAo8AxwAnqyql5I80h0/C3w/8LEkV4EvA39vF2uWJPXQ16/PrapzwLl1fWfX3D4P3DPY0iRJ2+E7RSWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqRF+BnuRkkoUkF5I83mP89iT/JckXkryU5OHBlypJ2syWgZ7kAPAEcD9wHHgoyfF1034G+HJVvRO4D/jXSW4bcK2SpE30c0I/AVyoqotV9TrwFPDAujkFfE+SAG8FvgFcGWilkqRN9RPoh4BX1rQXu31r/Srw/cCrwJeAD1bVtfV3lORUkvkk88vLyzdYsqStzC0tMXX+PGPPPsvU+fPMLS0NuyTtgX4CPT36al37x4EXgD8FvAv41SR//E1fVDVbVdNVNX3w4MFtFytpa3NLS5xaWODS6ioFXFpd5dTCgqF+E+gn0BeBu9a0D9M5ia/1MPDJ6rgA/B7wpwdToqTtOH3xIivXrv8BeeXaNU5fvDikirRX+gn054B7ktzdfaLzQeDpdXMuA38FIMkkcC/g3x5pCC6vrm6rX+24ZasJVXUlyaPAM8AB4MmqeinJI93xs8A/Bz6S5Et0LtH8XFV9bRfrlrSBI+PjXOoR3kfGx4dQjfbSloEOUFXngHPr+s6uuf0q8GODLU3SjThz7BinFhauu+wyMTbGmWPHhliV9oLvFJUaMzM5yey993J0fJwAR8fHmb33XmYmJ4ddmnZZXyd0SaNlZnLSAL8JeUKXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRBrokNcJAl6RGGOhSg+bmYGoKxsY6n+fmhl2R9oJv/ZcaMzcHp07BykqnfelSpw0wMzO8urT7PKFLjTl9+o/C/A0rK51+tc1Alxpz+fL2+tUOA11qzJEj2+tXOwx0qTFnzsDExPV9ExOdfrXNQJcaMzMDs7Nw9Cgknc+zsz4hejPwVS5Sg2ZmDPCbkSd0SWqEgS5JjTDQJakRfQV6kpNJFpJcSPJ4j/HHkrzQ/XgxydUkf2Lw5UqSNrJloCc5ADwB3A8cBx5KcnztnKr6V1X1rqp6F/DzwG9U1Td2o2BJUm/9nNBPABeq6mJVvQ48BTywyfyHgI8PojhJUv/6CfRDwCtr2ovdvjdJMgGcBD6xwfipJPNJ5peXl7dbqyRpE/0Eenr01QZz/xrwmxtdbqmq2aqarqrpgwcP9lujJKkP/QT6InDXmvZh4NUN5j6Il1skaSj6CfTngHuS3J3kNjqh/fT6SUluB/4S8OuDLVGS1I8t3/pfVVeSPAo8AxwAnqyql5I80h0/2536U8B/q6pv71q1kqQNpWqjy+G7a3p6uubn54fy2JI0qpI8X1XTvcZ8p6gkNcJAl6RGGOiS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5Jjegr0JOcTLKQ5EKSxzeYc1+SF5K8lOQ3BlumJGkrt2w1IckB4AngR4FF4LkkT1fVl9fMuQP4t8DJqrqc5O27VbAkqbd+TugngAtVdbGqXgeeAh5YN+engU9W1WWAqnptsGVKkrbST6AfAl5Z017s9q31fcDbkjyb5Pkk7+91R0lOJZlPMr+8vHxjFUuSeuon0NOjr9a1bwH+HPATwI8D/yTJ973pi6pmq2q6qqYPHjy47WIlSRvb8ho6nRP5XWvah4FXe8z5WlV9G/h2ks8C7wS+MpAqJUlb6ueE/hxwT5K7k9wGPAg8vW7OrwN/McktSSaAvwC8PNhSJUmb2fKEXlVXkjwKPAMcAJ6sqpeSPNIdP1tVLyf5r8AXgWvAr1XVi7tZuCTpeqlafzl8b0xPT9f8/PxQHluSRlWS56tquteY7xSVpEYY6JLUCANdkhphoGvH5paWmDp/nrFnn2Xq/HnmlpaGXZJ0U+rndejShuaWlji1sMDKtWsAXFpd5dTCAgAzk5PDLE266XhC146cvnjxu2H+hpVr1zh98eKQKpJuXga6duTy6uq2+iXtHgNdO3JkfHxb/ZJ2j4GuHTlz7BgTY9f/NZoYG+PMsWNDqki6efmkqHZkZnKS3/zEW5j9Z+NcfW2cA29f5W9/aJWZH7l92KVJNx1P6NqRuTn46GO3c3XpLVDh6tJb+OhjtzM3N+zKpJuPga4dOX0aVlau71tZ6fRL2lsGunbk8uXt9UvaPQa6duTIke31S9o9Brp25MwZmJi4vm9iotMvaW8Z6NqRmRmYnYWjRyHpfJ6d7fRL2lu+bFE7NjNjgEv7gSd0SWqEgS5JjTDQJakRBrokNcJAl6RG9BXoSU4mWUhyIcnjPcbvS/LNJC90Pz40+FIlSZvZ8mWLSQ4ATwA/CiwCzyV5uqq+vG7q/66qn9yFGiVJfejnhH4CuFBVF6vqdeAp4IHdLUuStF39BPoh4JU17cVu33o/lOQLST6T5Ad63VGSU0nmk8wvLy/fQLmSpI30E+jp0Vfr2p8DjlbVO4FfAT7V646qaraqpqtq+uDBg9urVJK0qX4CfRG4a037MPDq2glV9a2q+oPu7XPArUnuHFiVkqQt9RPozwH3JLk7yW3Ag8DTayck+d4k6d4+0b3frw+6WEnSxrZ8lUtVXUnyKPAMcAB4sqpeSvJId/ws8D7gA0muAN8BHqyq9ZdlJEm7KMPK3enp6Zqfnx/KY0vSqEryfFVN9xrznaKS1AgDXZIaYaBLUiMMdElqhIEuSY0w0CWpEQa6JDXCQJekRhjoktQIA12SGmGgS1IjDHRJaoSBLkmNMNAlqREGuiQ1wkCXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5JjTDQJakRfQV6kpNJFpJcSPL4JvP+fJKrSd43uBIlSf3YMtCTHACeAO4HjgMPJTm+wbx/CTwz6CIlSVvr54R+ArhQVRer6nXgKeCBHvN+FvgE8NoA65Mk9amfQD8EvLKmvdjt+64kh4CfAs4OrjRJ0nb0E+jp0Vfr2r8E/FxVXd30jpJTSeaTzC8vL/dboySpD7f0MWcRuGtN+zDw6ro508BTSQDuBN6b5EpVfWrtpKqaBWYBpqen139TkCTtQD+B/hxwT5K7gd8HHgR+eu2Eqrr7jdtJPgJ8en2YS5J215aBXlVXkjxK59UrB4Anq+qlJI90x71uLkn7QD8ndKrqHHBuXV/PIK+qv7PzsiRJ2zVS7xSdm4OpKRgb63yemxt2RZK0f/R1Qt8P5ubg7/6Da7z+nc73oEuXOm0YY2ZmuLVJ0n4wMif0Dz5+9bth/obXvzPGBx/f9JWSknTTGJlA//rv9y51o35JutmMThq+fXV7/ZJ0kxmZQP+Tj1yG8XWXV8avdvolSaMT6B/+wO3c+thXYPIPIQWTf8itj32FD3/g9mGXJkn7wsi8ymVmchIehdPv/TyXV1c5Mj7OmWPHOv2SpNEJdOiEugEuSb2NzCUXSdLmDHRJaoSBLkmNMNAlqREGuiQ1IlXD+Y+DkiwDl27wy+8EvjbAcobJtexPrayllXWAa3nD0ao62GtgaIG+E0nmq2p62HUMgmvZn1pZSyvrANfSDy+5SFIjDHRJasSoBvrssAsYINeyP7WyllbWAa5lSyN5DV2S9GajekKXJK1joEtSI/Z1oCd5MslrSV7cYDxJfjnJhSRfTPLuva6xH32s474k30zyQvfjQ3tdY7+S3JXkfyV5OclLST7YY86+35c+1zES+5LkLUl+J8kXumv5pz3m7Ps9gb7XMhL7ApDkQJLPJ/l0j7HB70lV7dsP4EeAdwMvbjD+XuAzQID3AL897JpvcB33AZ8edp19ruUdwLu7t78H+ApwfNT2pc91jMS+dP+c39q9fSvw28B7Rm1PtrGWkdiXbq3/CPiPverdjT3Z1yf0qvos8I1NpjwAfKw6fgu4I8k79qa6/vWxjpFRVV+tqs91b/8/4GXg0Lpp+35f+lzHSOj+Of9Bt3lr92P9qx32/Z5A32sZCUkOAz8B/NoGUwa+J/s60PtwCHhlTXuREf1HCfxQ98fMzyT5gWEX048kU8CfpXOKWmuk9mWTdcCI7Ev3R/sXgNeA/15VI7snfawFRmNffgn4x8C1DcYHviejHujp0TeK380/R+f3M7wT+BXgU0OuZ0tJ3gp8AviHVfWt9cM9vmRf7ssW6xiZfamqq1X1LuAwcCLJD66bMjJ70sda9v2+JPlJ4LWqen6zaT36drQnox7oi8Bda9qHgVeHVMsNq6pvvfFjZlWdA25NcueQy9pQklvphOBcVX2yx5SR2Jet1jFq+wJQVf8XeBY4uW5oJPZkrY3WMiL78sPAX0/yf4CngL+c5D+smzPwPRn1QH8aeH/32eL3AN+sqq8Ou6jtSvK9SdK9fYLOvnx9uFX11q3z3wMvV9W/2WDavt+XftYxKvuS5GCSO7q3/xjwV4HfXTdt3+8J9LeWUdiXqvr5qjpcVVPAg8D/rKq/uW7awPdkX/8n0Uk+TucZ7TuTLAK/SOdJEqrqLHCOzjPFF4AV4OHhVLq5PtbxPuADSa4A3wEerO7T4PvQDwN/C/hS9zonwC8AR2Ck9qWfdYzKvrwD+GiSA3TC7T9V1aeTPAIjtSfQ31pGZV/eZLf3xLf+S1IjRv2SiySpy0CXpEYY6JLUCANdkhphoEtSIwx0SWqEgS5Jjfj/vzdE0+2aS1wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#mathematica\n",
    "    #run 1: 645s = 10,8 min\n",
    "#python\n",
    "    #run 1: 884s = 14,7 min\n",
    "    #run 2: ~940s = ~15,7 min\n",
    "\n",
    "subsetNetwork22 = randomNetwork([2,2], 10)\n",
    "  \n",
    "start = time() #Optional\n",
    "\n",
    "pointsX = list(range(1,5))\n",
    "pointsBoundRand = [boundRand(4, 10, n) for n in pointsX]\n",
    "pointsAverageCost = [subsetTrainingAvg(subsetNetwork22[0], subsetNetwork22[1], subsetNetwork22[2], 1.5, 0.1, 1000, 20, n, alertIt=20) for n in pointsX]\n",
    "\n",
    "print(time() - start) #Optional\n",
    "\n",
    "plt.plot(pointsX, pointsBoundRand, 'co')\n",
    "plt.plot(pointsX, pointsAverageCost, 'bo')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Subset training for a 3-3-3 network | D = 8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "#mathematica\n",
    "    #run 1: 45744s > 12h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4 Robustness to noisy data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "noiseTrainingData = randomTrainingData(randomQubitUnitary(2), 100)\n",
    "noiseNoisyData = [[randomQubitState(2), randomQubitState(2)] for i in range(100)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Noisy data for a 2-2 network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 165.34 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 163.93 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 166.27 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 190.67 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 230.14 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 191.08 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 179.44 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 184.26 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 240.92 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 226.44 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 185.44 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 198.04 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 198.08 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 247.36 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 246.97 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 271.49 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 199.59 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 175.04 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 171.81 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 166.65 seconds\n",
      "Trained 300 rounds for a [2, 2] network and 100 training pairs in 167.91 seconds\n",
      "4174.151624202728\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAUtElEQVR4nO3df5Bd533X8ffHsly6KVUcvA2pfuwKEE3VKkmTRaQQaECEyqFGDXQGuVtSQpgdMXVJGQbqsDOdejI7Q4YfUZk61eykIi3eiYZp0kQKblxGQM3QptWqdSzLjlLVjn5UBm8SUEjUwVLy5Y+9Mleru9q72ru6e8++XzN3fM9znnPP91nJn3t09pznpKqQJA2+u/pdgCSpNwx0SWoIA12SGsJAl6SGMNAlqSHu7teO77vvvhodHe3X7iVpIJ08efJLVTXcaV3fAn10dJTZ2dl+7V6SBlKSc4ut85SLJDWEgS5JDWGgS1JDGOiS1BAGuiQ1xJKBnuRwkpeSPLPI+iT5t0nOJnk6yZt7X+a8mVMzjB4c5a5H7mL04Cgzp2bu2PYr3bckrbZujtA/Cuy9xfr7gR2t1wTwCysv62Yzp2aYODbBucvnKIpzl88xcWyi62Bdyfa92He/vkj6te9+fQH6xav1bMlAr6onga/coss+4Jdr3meBVyd5Xa8KvG7y+CRXrl65oe3K1StMHp9c9e1Xsm2/v0j6se9+fQGudL8r2fdK+UWkXujFOfTNwIW25YuttpskmUgym2R2bm5uWTs5f/n8stp7uf1Ktu3XF0k/992vL8CV/rwG+YtIgt4Eejq0dXxqRlVNV9VYVY0ND3e8c3VR2zZtW1Z7L7dfybb9+iLp57779QW40p/XoH4RSdf1ItAvAlvblrcAl3rwuTeY2jPF0MahG9qGNg4xtWdq1bdfybb9+iLp57779QW40p/XoH4RSdf1ItCPAu9uXe3yVuByVb3Yg8+9wfiucaYfmGZk0wghjGwaYfqBacZ3ja/69ivZtl9fJP3cd7++AFf68xrULyLpFVV1yxfwMeBF4CrzR+PvBQ4AB1rrAzwK/AFwChhb6jOrire85S21Xjz29GM18qGRys+mRj40Uo89/dgd2baf+77dbR97+rEamhoqfpZXXkNTQ8vafiU13+6+Rz40csN2118jHxpZ1f1q/QFma5FcTfXpIdFjY2PlbIvqZObUDJPHJzl/+TzbNm1jas9U1/8S69e+r59Dbz/tMrRxqOt/ya1kzP38eenOS3KyqsY6rjPQpd7oR7Cu9ItEg8dAlxpq9OAo5y7fPD32yKYRvvhTX7zzBWnV3SrQnctFGmBeIaN2Bro0wLxCRu0MdGmArfRSTTWLgS4NsJXen6Fm8ZeikjRA/KWoJK0DBrq0jjltb7Pc3e8CJPXHwpuSrs8QCXgOfkB5hC6tU07b2zwGurROeVNS8xjo0jrlTUnNY6BL65Q3JTWPgS6tU96U1DzeWCRJA2TFNxYl2ZvkTJKzSR7usP7eJL+a5Okkv5Pke1datCRpeZYM9CQbmH/E3P3ATuDBJDsXdPvnwFNV9Qbg3cDP9bpQSdKtdXOEvhs4W1XPV9XLwBFg34I+O4HjAFX1eWA0yWt7Wqkk6Za6CfTNwIW25YuttnafA/4WQJLdwAiwZeEHJZlIMptkdm5u7vYqliR11E2gp0Pbwt+k/gvg3iRPAT8J/B5w7aaNqqaraqyqxoaHh5ddrCRpcd3M5XIR2Nq2vAW41N6hqr4KvAcgSYAXWi9J0h3SzRH6CWBHku1J7gH2A0fbOyR5dWsdwD8AnmyFvCTpDlnyCL2qriV5CHgC2AAcrqrTSQ601h8Cvhv45STfAJ4F3ruKNUuSOuhq+tyqehx4fEHbobb3vwXs6G1pkta6mVMzTB6f5Pzl82zbtI2pPVPeadpHzocu6bY4n/ra41wukm6L86mvPQa6pNvifOprj4Eu6bY4n/raY6BLui3Op772GOiSbovzqa89zocuSQNkxfOhS5LWPgNdkhrCQJekhjDQJakhDHRJaggDXZIawkCXpIYw0CWpIQx0SWqIrgI9yd4kZ5KcTfJwh/WbkhxL8rkkp5O8p/elSpJuZclAT7IBeBS4H9gJPJhk54JuPwE8W1VvBN4O/Ou2Z4xKku6Abo7QdwNnq+r5qnoZOALsW9CngD+eJMC3AV8BrvW0UknSLXUT6JuBC23LF1tt7X6e+QdFXwJOAe+rqm8u/KAkE0lmk8zOzc3dZsmSpE66CfR0aFs4ReMPAk8B3wm8Cfj5JN9+00ZV01U1VlVjw8PDyy5WkrS4bgL9IrC1bXkL80fi7d4DfKLmnQVeAF7fmxIlSd3oJtBPADuSbG/9onM/cHRBn/PAHoAkrwW+C3i+l4VKkm7t7qU6VNW1JA8BTwAbgMNVdTrJgdb6Q8AHgI8mOcX8KZqfrqovrWLdkqQFlgx0gKp6HHh8QduhtveXgL/e29IkScvhnaKS1BAGuiQ1hIEuSQ1hoEtSQxjoktQQBrokNYSBLkkNYaBLUkMY6JLUEAa6JDWEgS5JDWGgS1JDGOiS1BAGuiQ1hIEuqS9mTs0wenCUux65i9GDo8ycmul3SQOvq/nQJamXZk7NMHFsgitXrwBw7vI5Jo5NADC+a7yfpQ20ro7Qk+xNcibJ2SQPd1j/T5M81Xo9k+QbSV7T+3IlNcHk8clXwvy6K1evMHl8sk8VNcOSgZ5kA/AocD+wE3gwyc72PlX1L6vqTVX1JuD9wG9U1VdWo2BJg+/85fPLald3ujlC3w2crarnq+pl4Aiw7xb9HwQ+1oviJDXTtk3bltWu7nQT6JuBC23LF1ttN0kyBOwFPr7I+okks0lm5+bmllurpIaY2jPF0MahG9qGNg4xtWeqTxU1QzeBng5ttUjfB4D/vtjplqqarqqxqhobHh7utkZJDTO+a5zpB6YZ2TRCCCObRph+YNpfiK5QN1e5XAS2ti1vAS4t0nc/nm6R1IXxXeMGeI91c4R+AtiRZHuSe5gP7aMLOyXZBPwA8KnelihJ6saSR+hVdS3JQ8ATwAbgcFWdTnKgtf5Qq+u7gF+vqq+vWrWSpEWlarHT4atrbGysZmdn+7JvSRpUSU5W1Vindd76L0kNYaBLUkMY6JLUEAa6JDWEgS5JDWGgS1JDGOiS1BAGuiQ1hIEuSQ1hoEtSQxjoktQQBrokNYSBLkkNYaBLUkMY6JLUEAa6JDVEV4GeZG+SM0nOJnl4kT5vT/JUktNJfqO3ZUqSlrLkI+iSbAAeBd7B/AOjTyQ5WlXPtvV5NfBhYG9VnU/yHatVsCSps26O0HcDZ6vq+ap6GTgC7FvQ50eBT1TVeYCqeqm3ZUqSltJNoG8GLrQtX2y1tfuzwL1J/muSk0ne3asCJUndWfKUC5AObQufLH038BZgD/CtwG8l+WxVfeGGD0omgAmAbdu2Lb9aSdKiujlCvwhsbVveAlzq0OczVfX1qvoS8CTwxoUfVFXTVTVWVWPDw8O3W7MkqYNuAv0EsCPJ9iT3APuBowv6fAr4S0nuTjIE/Hngud6WKkm6lSVPuVTVtSQPAU8AG4DDVXU6yYHW+kNV9VySzwBPA98EPlJVz6xm4ZKkG6Vq4enwO2NsbKxmZ2f7sm9JGlRJTlbVWKd13ikqSQ1hoEtSQxjoktQQBrokNYSBLkkNYaBLUkMY6JLUEAa6JDWEgS5JDWGgS1JDGOiS1BAGuiQ1hIEuSQ1hoEtSQxjoktQQBrokNURXgZ5kb5IzSc4mebjD+rcnuZzkqdbrZ3pfqiTpVpZ8BF2SDcCjwDuYfxj0iSRHq+rZBV3/W1X90CrUKEnqQjdH6LuBs1X1fFW9DBwB9q1uWZKk5eom0DcDF9qWL7baFvr+JJ9L8mtJvqcn1UmSurbkKRcgHdoWPln6d4GRqvpakncCnwR23PRByQQwAbBt27ZllipJupVujtAvAlvblrcAl9o7VNVXq+prrfePAxuT3Lfwg6pquqrGqmpseHh4BWVLkhbqJtBPADuSbE9yD7AfONreIcmfTJLW+92tz/1yr4uVJC1uyVMuVXUtyUPAE8AG4HBVnU5yoLX+EPAjwD9Mcg34I2B/VS08LSNJWkXpV+6OjY3V7OxsX/YtSYMqycmqGuu0zjtFJakhDHRJaggDXZIawkCXpIYw0CWpIQx0SWoIA13SwJk5NcPowVHueuQuRg+OMnNqpt8lrQndzOUiSWvGzKkZJo5NcOXqFQDOXT7HxLEJAMZ3jfeztL7zCF3SQJk8PvlKmF935eoVJo9P9qmitcNAlzRQzl8+v6z29cRAlzRQtm3qPPX2Yu3riYEuaaBM7ZliaOPQDW1DG4eY2jPVp4rWDgNd0kAZ3zXO9APTjGwaIYSRTSNMPzC97n8hCs62KEkDxdkWJWkdMNAlqSG6CvQke5OcSXI2ycO36PfnknwjyY/0rkRJUjeWDPQkG4BHgfuBncCDSXYu0u+DzD+qTpJ0h3VzhL4bOFtVz1fVy8ARYF+Hfj8JfBx4qYf1SZK61E2gbwYutC1fbLW9Islm4F3Aod6VJklajm4CPR3aFl7reBD46ar6xi0/KJlIMptkdm5urtsaJUld6Ga2xYvA1rblLcClBX3GgCNJAO4D3pnkWlV9sr1TVU0D0zB/HfrtFi1Julk3gX4C2JFkO/CHwH7gR9s7VNX26++TfBT49MIwlyStriUDvaquJXmI+atXNgCHq+p0kgOt9Z43l6Q1oKsHXFTV48DjC9o6BnlV/b2VlyVJWi7vFJWkhjDQJakhDHRJaggDXZIawkCXpIYw0CWpIQx0SWoIA12SGsJAl6SGMNAlqSEMdElqCANdkhrCQJekhjDQJakhDHRJaggDXZIawkCXpIboKtCT7E1yJsnZJA93WL8vydNJnkoym+RtvS9VknQrSz6CLskG4FHgHcBF4ESSo1X1bFu348DRqqokbwD+A/D61ShYktRZN0fou4GzVfV8Vb0MHAH2tXeoqq9VVbUWXwUUkqQ7qptA3wxcaFu+2Gq7QZJ3Jfk88B+Bv9/pg5JMtE7JzM7Nzd1OvZK0IjOnZhg9OMpdj9zF6MFRZk7N9Luknukm0NOh7aYj8Kr61ap6PfDDwAc6fVBVTVfVWFWNDQ8PL69SSVqhmVMzTByb4NzlcxTFucvnmDg20ZhQ7ybQLwJb25a3AJcW61xVTwJ/Osl9K6xNknpq8vgkV65euaHtytUrTB6f7FNFvdVNoJ8AdiTZnuQeYD9wtL1Dkj+TJK33bwbuAb7c62IlaSXOXz6/rPZBs+RVLlV1LclDwBPABuBwVZ1OcqC1/hDwt4F3J7kK/BHwd9p+SSpJa8K2Tds4d/lcx/YmWDLQAarqceDxBW2H2t5/EPhgb0uTpN6a2jPFxLGJG067DG0cYmrPVB+r6h3vFJW0bozvGmf6gWlGNo0QwsimEaYfmGZ813i/S+uJ9OvMyNjYWM3OzvZl35I0qJKcrKqxTus8QpekhjDQJakhDHRJaggDXZKWYS1PHdDVZYuSpP8/dcD1yx6vTx0ArIkrZTxCl6QurfWpAwx0SerSWp86wECXpC4tNkXAWpk6wECXpC5N7ZliaOPQDW1raeoAA12SurTWpw7w1n9JGiDe+i9Ja8BqX8PudeiSdAfciWvYPUKXpDvgTlzD3lWgJ9mb5EySs0ke7rB+PMnTrddvJnljzyqUpAa4E9ewLxnoSTYAjwL3AzuBB5PsXNDtBeAHquoNwAeA6Z5VKEkNcCeuYe/mCH03cLaqnq+ql4EjwL72DlX1m1X1v1qLnwW29KxCSWqAO3ENezeBvhm40LZ8sdW2mPcCv9ZpRZKJJLNJZufm5rqvUpIG3J24hr2bq1zSoa3jxetJ/grzgf62TuuraprW6ZixsbH+XAAvSX0yvmt8VW9C6ibQLwJb25a3AJcWdkryBuAjwP1V9eXelCdJ6lY3p1xOADuSbE9yD7AfONreIck24BPA362qL/S+TEnSUpY8Qq+qa0keAp4ANgCHq+p0kgOt9YeAnwH+BPDhJADXFrs1VZK0OpzLRZIGiHO5SNI60Lcj9CRzwLnb3Pw+4Es9LGcQOOb1wTGvDysZ80hVDXda0bdAX4kks+vtHL1jXh8c8/qwWmP2lIskNYSBLkkNMaiBvh4n/3LM64NjXh9WZcwDeQ5dknSzQT1ClyQtYKBLUkMMXKAv9fSkJkiyNcl/SfJcktNJ3tdqf02S/5Tk91v/vbfftfZSkg1Jfi/Jp1vLTR/vq5P8SpLPt/6sv38djPkft/5OP5PkY0n+WNPGnORwkpeSPNPWtugYk7y/lWdnkvzgSvY9UIHe5dOTmuAa8E+q6ruBtwI/0Rrnw8DxqtoBHG8tN8n7gOfalps+3p8DPlNVrwfeyPzYGzvmJJuBfwSMVdX3Mj831H6aN+aPAnsXtHUcY+v/6/3A97S2+XAr527LQAU6XTw9qQmq6sWq+t3W+//D/P/om5kf6y+1uv0S8MP9qbD3kmwB/gbzUzBf1+Txfjvwl4FfBKiql6vqf9PgMbfcDXxrkruBIean4m7UmKvqSeArC5oXG+M+4EhV/d+qegE4y3zO3ZZBC/TlPj1p4CUZBb4P+G3gtVX1IsyHPvAd/aus5w4C/wz4Zltbk8f7p4A54N+1TjN9JMmraPCYq+oPgX8FnAdeBC5X1a/T4DG3WWyMPc20QQv0rp+e1ARJvg34OPBTVfXVftezWpL8EPBSVZ3sdy130N3Am4FfqKrvA77O4J9quKXWeeN9wHbgO4FXJfmx/lbVdz3NtEEL9K6entQESTYyH+YzVfWJVvP/TPK61vrXAS/1q74e+4vA30zyReZPo/3VJI/R3PHC/N/li1X1263lX2E+4Js85r8GvFBVc1V1lfmH4vwFmj3m6xYbY08zbdACfcmnJzVB5p8S8ovAc1X1b9pWHQV+vPX+x4FP3enaVkNVvb+qtlTVKPN/pv+5qn6Mho4XoKr+B3AhyXe1mvYAz9LgMTN/quWtSYZaf8f3MP/7oSaP+brFxngU2J/kW5JsB3YAv3Pbe6mqgXoB7wS+APwBMNnvelZpjG9j/p9dTwNPtV7vZP6pUMeB32/99zX9rnUVxv524NOt940eL/AmYLb15/xJ4N51MOZHgM8DzwD/HviWpo0Z+BjzvyO4yvwR+HtvNUZgspVnZ5h/JvNt79tb/yWpIQbtlIskaREGuiQ1hIEuSQ1hoEtSQxjoktQQBrokNYSBLkkN8f8AEGdwEjeZp0sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#mathematica\n",
    "    #run 1: 1659s = 27,7 min\n",
    "#python\n",
    "    #run 1: 2284s = 38,1 min\n",
    "\n",
    "start = time() #Optional\n",
    "\n",
    "noiseNetwork22 = randomNetwork([2,2], 0)\n",
    "noisePlotlist22 = noisyDataTraining(noiseNetwork22[0], noiseNetwork22[1], noiseTrainingData.copy(), noiseNoisyData.copy(), 1, 0.1, 300, 100, 5)\n",
    "\n",
    "print(time() - start) #Optional\n",
    "\n",
    "plt.plot(noisePlotlist22[0], noisePlotlist22[1], 'go')\n",
    "plt.show() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Noisy data for a 2-3-2 network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 544.03 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 537.44 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 538.77 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 539.67 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 570.2 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 939.64 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1095.14 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1104.58 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1140.37 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1094.75 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1130.88 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1082.0 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1091.39 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1076.31 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 1129.11 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 590.94 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 558.69 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 567.14 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 572.52 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 574.54 seconds\n",
      "Trained 300 rounds for a [2, 3, 2] network and 100 training pairs in 567.69 seconds\n",
      "17063.1174762249\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAUdElEQVR4nO3df5Bd533X8fdHslzYlCoO3oZUP3YFqE3VKk7DoqRQqEENkdMKNdAZZLakE8LsGOqSMAzEYYd2PJ2dgeHHqBCnmp1UTQs70TBJmkjFrcsIqIHSVuvWsSw7SoVt/UCh3iRFoVEHS/GXP/bKXK32x13t3b17z75fMzva85zn3P0+I+lznz33nPOkqpAk9b9NvS5AktQdBrokNYSBLkkNYaBLUkMY6JLUEHf16gffe++9NTw83KsfL0l96amnnvpSVQ3Ot69ngT48PMz09HSvfrwk9aUkFxba5ykXSWoIA12SGsJAl6SGMNAlqSEMdElqiCUDPcmxJC8neXaB/Unyr5KcT/JMkrd1v8xZU2emGD4yzKZHNzF8ZJipM1NrdnyvjpWkTnUyQ/84cGCR/Q8Au1tfY8BPr7ys202dmWLs5BgXrl6gKC5cvcDYybGOw3Elx/fq2JvH+yYmqRNLBnpVPQl8ZZEuh4Cfr1m/Drw+yZu6VeBN46fGuXb92i1t165fY/zU+Kof36tjfRPrrzeSfq1bzdGNc+jbgEtt25dbbbdJMpZkOsn0zMzMsn7IxasXl9XezeN7daxvYmv7RrIS/Vq3mqUbgZ552uZdNaOqJqtqpKpGBgfnvXN1QTu37lxWezeP79WxvonNWos3kpvudKbc67ol6E6gXwZ2tG1vB6504XVvMbF/goEtA7e0DWwZYGL/xKof36tjfRNbur1bx8LKZsq9rFu6qRuBfgJ4b+tql3cAV6vqi1143VuM7h1l8uAkQ1uHCGFo6xCTBycZ3Tu66sf36ljfxJZu79axsLKZci/r9vy7bspSa4om+QRwP3Av8LvATwBbAKrqaJIAH2H2SphrwPuqasmnbo2MjJQP51ra1Jkpxk+Nc/HqRXZu3cnE/omO38RWenwvjr05S24P1oEtAx29Ca7kWIBNj26i5jlbGMKrP/Hquqx7pWNW/0nyVFWNzLuvV4tEG+haSK/ehIaPDHPh6u0PshvaOsRLH3xpXdbdy5rVGwa61IF+nO326rcK9c5ige6t/1LLSj+n6YWVnH/36prm6dkCF9J6NLp3dF0H+FwT+yfmnWV38gG0V9c0jzN0qY+t5LeKlV5do/XHGbrU5+70t4qVzO61PjlDlzaofvzMQIvzKhdJ6iNe5SJJG4CBLkkNYaBLUkMY6JLUEAa6JDWEgS7pjvno3vXFG4sk3ZG5D/e6uSAI4LXsPeIMXdId8eFe64+BLumO+HCv9aejQE9yIMm5JOeTPDLP/nuS/EKSZ5L8ZpLv7H6pktYTH+61/iwZ6Ek2A48BDwB7gAeT7JnT7R8BT1fVW4D3Aj/V7UIlrS8rXfNW3dfJDH0fcL6qXqiqV4DjwKE5ffYApwCq6vPAcJI3drVSSeuKD/dafzq5ymUbcKlt+zLw9jl9Pgf8FeC/JtkHDAHbmV1U+jVJxoAxgJ07/bVM6nf9tiBI03UyQ888bXMf0fhPgHuSPA38GPDbwI3bDqqarKqRqhoZHBxcdrGSpIV1MkO/DOxo294OXGnvUFVfBd4HkCTAi60vSdIa6WSGfhrYnWRXkruBw8CJ9g5JXt/aB/C3gCdbIS9JWiNLztCr6kaSh4EngM3Asao6m+Sh1v6jwLcDP5/k68BzwPtXsWZJ0jw6uvW/qh4HHp/TdrTt+/8O7O5uaZKk5fBOUUlqCANdkhrCQJekhjDQJakhDHRJaggDXZIawkCXpIYw0CWpIQx0SWoIA12SGsJAl6SGMNAlqSEMdElqCANdkhrCQJekhjDQJakhOgr0JAeSnEtyPskj8+zfmuRkks8lOZvkfd0vVZK0mCUDPclm4DHgAWAP8GCSPXO6/SjwXFXdB9wP/Iu2NUYlSWugkxn6PuB8Vb1QVa8Ax4FDc/oU8EeSBPhG4CvAja5WKklaVCeBvg241LZ9udXW7iPMLhR9BTgDfKCqXp37QknGkkwnmZ6ZmbnDkiVJ8+kk0DNPW83ZfhfwNPAtwFuBjyT5ptsOqpqsqpGqGhkcHFx2sZKkhXUS6JeBHW3b25mdibd7H/DpmnUeeBF4c3dKlCR1opNAPw3sTrKr9UHnYeDEnD4Xgf0ASd4IfBvwQjcLlSQt7q6lOlTVjSQPA08Am4FjVXU2yUOt/UeBnwQ+nuQMs6doPlRVX1rFuiVJcywZ6ABV9Tjw+Jy2o23fXwH+UndLkyQth3eKSlJDGOiS1BAGuiQ1hIEuSQ1hoEtSQxjoktQQBrokNYSBLkkNYaBLUkMY6JLUEAa6JDWEgS5JDWGgS1JDGOiS1BAGuiQ1REeBnuRAknNJzid5ZJ79/yDJ062vZ5N8Pckbul+uJGkhSwZ6ks3AY8ADwB7gwSR72vtU1T+rqrdW1VuBDwO/WlVfWY2CJUnz62SGvg84X1UvVNUrwHHg0CL9HwQ+0Y3iJEmd6yTQtwGX2rYvt9puk2QAOAB8auWlSZKWo5NAzzxttUDfg8B/W+h0S5KxJNNJpmdmZjqtUZLUgU4C/TKwo217O3Blgb6HWeR0S1VNVtVIVY0MDg52XqUkaUmdBPppYHeSXUnuZja0T8ztlGQr8L3AZ7tboiSpE3ct1aGqbiR5GHgC2Awcq6qzSR5q7T/a6voe4Feq6murVq0kaUGpWuh0+OoaGRmp6enpnvxsSepXSZ6qqpH59nmnqCQ1hIEuSQ1hoEtSQxjoktQQBrqknpg6M8XwkWE2PbqJ4SPDTJ2Z6nVJfW/JyxYlqdumzkwxdnKMa9evAXDh6gXGTo4BMLp3tJel9TVn6JLW3Pip8dfC/KZr168xfmq8RxU1g4Euac1dvHpxWe3qjIEuac3t3LpzWe3qjIEuac1N7J9gYMvALW0DWwaY2D/Ro4qawUCXtOZG944yeXCSoa1DhDC0dYjJg5N+ILpCPstFkvqIz3KRpA3AQJekhjDQJakhDHRJaoiOAj3JgSTnkpxP8sgCfe5P8nSSs0l+tbtlSpKWsuSzXJJsBh4D3snsgtGnk5yoqufa+rwe+ChwoKouJvnm1SpYkjS/Tmbo+4DzVfVCVb0CHAcOzenz14FPV9VFgKp6ubtlSpKW0kmgbwMutW1fbrW1+1bgniT/OclTSd473wslGUsynWR6ZmbmziqWJM2rk0DPPG1z70a6C/hTwPcD7wL+cZJvve2gqsmqGqmqkcHBwWUXK0laWCfPQ78M7Gjb3g5cmafPl6rqa8DXkjwJ3Ad8oStVSpKW1MkM/TSwO8muJHcDh4ETc/p8FvhzSe5KMgC8HXi+u6VKkhaz5Ay9qm4keRh4AtgMHKuqs0keau0/WlXPJ/ll4BngVeBjVfXsahYuSbqVD+eSpD7iw7kkaQMw0CWpIQx0SWoIA11S35k6M8XwkWE2PbqJ4SPDTJ2Z6nVJ60In16FL0roxdWaKsZNjXLt+DYALVy8wdnIMYMMvYecMXVJfGT81/lqY33Tt+jXGT433qKL1w0CX1FcuXr24rPaNxECX1Fd2bt25rPaNxECX1Fcm9k8wsGXglraBLQNM7J/oUUXrh4Euqa+M7h1l8uAkQ1uHCGFo6xCTByc3/Aei4K3/ktRXvPVfkjYAA12SGsJAl6SGMNAlqSEMdElqiI4CPcmBJOeSnE/yyDz7709yNcnTra8f736pkqTFLPlwriSbgceAdzK7GPTpJCeq6rk5Xf9LVf3AKtQoSepAJzP0fcD5qnqhql4BjgOHVrcsSdJydRLo24BLbduXW21zfXeSzyX5pSTfMd8LJRlLMp1kemZm5g7KlSQtpJNAzzxtc28v/S1gqKruA/418Jn5XqiqJqtqpKpGBgcHl1epJGlRnQT6ZWBH2/Z24Ep7h6r6alX9fuv7x4EtSe7tWpWSpCV1Euingd1JdiW5GzgMnGjvkOSPJUnr+32t1/1yt4uVJC1syatcqupGkoeBJ4DNwLGqOpvkodb+o8APAX87yQ3gD4DD1aunfknSBuXTFiWpj/i0RUnaAAx0SWoIA12SGsJAl6SGMNAlqSEMdElqCANdkhrCQJekhjDQJakhDHRJaggDXZIawkCXpIYw0CWpIQx0SWoIA12SGsJAl6SG6CjQkxxIci7J+SSPLNLvTyf5epIf6l6JkqROLBnoSTYDjwEPAHuAB5PsWaDfP2V2qTpJ0hrrZIa+DzhfVS9U1SvAceDQPP1+DPgU8HIX65MkdaiTQN8GXGrbvtxqe02SbcB7gKOLvVCSsSTTSaZnZmaWW6skaRGdBHrmaZu7svQR4ENV9fXFXqiqJqtqpKpGBgcHO61RktSBuzrocxnY0ba9Hbgyp88IcDwJwL3Au5PcqKrPdKVKSdKSOpmhnwZ2J9mV5G7gMHCivUNV7aqq4aoaBj4J/B3DXNJ6NHVmiuEjw2x6dBPDR4aZOjPV65K6ZskZelXdSPIws1evbAaOVdXZJA+19i963lyS1oupM1OMnRzj2vVrAFy4eoGxk2MAjO4d7WVpXZGquafD18bIyEhNT0/35GdL2piGjwxz4eqF29qHtg7x0gdfWvuC7kCSp6pqZL593ikqacO4ePXistr7jYEuacPYuXXnstr7jYEuacOY2D/BwJaBW9oGtgwwsX+iRxV1l4EuacMY3TvK5MFJhrYOEcLQ1iEmD0424gNR8ENRSeorfigqSRuAgS5JDWGgS1JDGOiS1BAGuiQ1hIEuSQ1hoEtSQxjoktQQBrokNYSBLkkNYaBLUkN0FOhJDiQ5l+R8kkfm2X8oyTNJnk4yneR7ul+qJGkxSy5Bl2Qz8BjwTmYXjD6d5ERVPdfW7RRwoqoqyVuAfwe8eTUKliTNr5MZ+j7gfFW9UFWvAMeBQ+0dqur36/8/tvF1QG8e4ShJG1gngb4NuNS2fbnVdosk70nyeeDfA39zvhdKMtY6JTM9MzNzJ/VKkhbQSaBnnrbbZuBV9QtV9WbgB4GfnO+FqmqyqkaqamRwcHB5lUqSFtVJoF8GdrRtbweuLNS5qp4E/kSSe1dYmyRpGToJ9NPA7iS7ktwNHAZOtHdI8ieTpPX924C7gS93u1hJ0sKWvMqlqm4keRh4AtgMHKuqs0keau0/CvxV4L1JrgN/APy16tXadpK0QbmmqCT1EdcUlaQNwECXpIYw0CWpIQx0SWoIA12SGsJAl6RlmDozxfCRYTY9uonhI8NMnZnqdUmvWfI6dEnSrKkzU4ydHOPa9WsAXLh6gbGTYwCM7h3tZWmAM3RJ6tj4qfHXwvyma9evMX5qvEcV3cpAl6QOXbx6cVnta81Al6QO7dy6c1nta81Al6QOTeyfYGDLwC1tA1sGmNg/0aOKbmWgS1KHRveOMnlwkqGtQ4QwtHWIyYOT6+IDUfDhXJLUV3w4lyRtAAa6JDVER4Ge5ECSc0nOJ3lknv2jSZ5pff1akvu6X6ok9bfVvst0yTtFk2wGHgPeyez6oqeTnKiq59q6vQh8b1X9XpIHgEng7V2tVJL62FrcZdrJDH0fcL6qXqiqV4DjwKH2DlX1a1X1e63NX2d2IWlJUsta3GXaSaBvAy61bV9utS3k/cAvraQoSWqatbjLtJNAzzxt817rmOQvMBvoH1pg/1iS6STTMzMznVcpSX1uLe4y7STQLwM72ra3A1fmdkryFuBjwKGq+vJ8L1RVk1U1UlUjg4ODd1KvJPWltbjLtJNAPw3sTrIryd3AYeBEe4ckO4FPA3+jqr7QteokqSHW4i7TJa9yqaobSR4GngA2A8eq6mySh1r7jwI/DvxR4KNJAG4sdCeTJG1Uo3tHV/UxAd76L0l9xFv/JWkDMNAlqSEMdElqCANdkhqiZx+KJpkBLtzh4fcCX+piOf3AMW8MjnljWMmYh6pq3ht5ehboK5FkeqNdFumYNwbHvDGs1pg95SJJDWGgS1JD9GugT/a6gB5wzBuDY94YVmXMfXkOXZJ0u36doUuS5jDQJakh+i7Ql1qwugmS7Ejyn5I8n+Rskg+02t+Q5D8k+Z3Wn/f0utZuSrI5yW8n+cXWdtPH+/okn0zy+dbf9XdvgDH/vda/6WeTfCLJH2ramJMcS/Jykmfb2hYcY5IPt/LsXJJ3reRn91Wgty1Y/QCwB3gwyZ7eVrUqbgB/v6q+HXgH8KOtcT4CnKqq3cCp1naTfAB4vm276eP9KeCXq+rNwH3Mjr2xY06yDfi7wEhVfSezj+M+TPPG/HHgwJy2ecfY+n99GPiO1jEfbeXcHemrQKeDBauboKq+WFW/1fr+/zD7H30bs2P9uVa3nwN+sDcVdl+S7cD3M7vq1U1NHu83AX8e+BmAqnqlqv43DR5zy13AH05yFzDA7OpnjRpzVT0JfGVO80JjPAQcr6r/W1UvAueZzbk70m+BvtwFq/tekmHgu4DfAN5YVV+E2dAHvrl3lXXdEeAfAq+2tTV5vH8cmAF+tnWa6WNJXkeDx1xV/xP458BF4IvA1ar6FRo85jYLjbGrmdZvgd7xgtVNkOQbgU8BH6yqr/a6ntWS5AeAl6vqqV7XsobuAt4G/HRVfRfwNfr/VMOiWueNDwG7gG8BXpfkh3tbVc91NdP6LdA7WrC6CZJsYTbMp6rq063m303yptb+NwEv96q+LvuzwF9O8hKzp9H+YpJ/S3PHC7P/li9X1W+0tj/JbMA3eczfB7xYVTNVdZ3ZdYj/DM0e800LjbGrmdZvgb7kgtVNkNmFWX8GeL6q/mXbrhPAj7S+/xHgs2td22qoqg9X1faqGmb27/Q/VtUP09DxAlTV/wIuJfm2VtN+4DkaPGZmT7W8I8lA69/4fmY/H2rymG9aaIwngMNJviHJLmA38Jt3/FOqqq++gHcDXwD+BzDe63pWaYzfw+yvXc8AT7e+3s3sQtyngN9p/fmGXte6CmO/H/jF1veNHi/wVmC69ff8GeCeDTDmR4HPA88C/wb4hqaNGfgEs58RXGd2Bv7+xcYIjLfy7BzwwEp+trf+S1JD9NspF0nSAgx0SWoIA12SGsJAl6SGMNAlqSEMdElqCANdkhri/wG6oLwOE+HsLgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#mathematica\n",
    "    #run 1: 6256s = 104 min\n",
    "#python\n",
    "    #run 1: 7649s = 127 min\n",
    "\n",
    "start = time() #Optional\n",
    "\n",
    "noiseNetwork232 = randomNetwork([2,3,2], 0)\n",
    "noisePlotlist232 = noisyDataTraining(noiseNetwork232[0], noiseNetwork232[1], noiseTrainingData.copy(), noiseNoisyData.copy(), 1, 0.1, 300, 100, 5)\n",
    "\n",
    "print(time() - start) #Optional\n",
    "\n",
    "plt.plot(noisePlotlist232[0], noisePlotlist232[1], 'go')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
